Monthly Archive for December, 2012

Running Benchmarks With the Phoronix Test Suite!

Benchmarking software can be invaluable when testing new hardware/software configurations. The Phoronix Test Suite is a collection of open source software benchmarks that are fairly easy to use and the results are presented in a such a way that is easy to understand; even if you don't understand the tests that were run. Today I'll give a brief run down of how to install the test suite and run the benchmarks.

Phoronix can be used on almost all operating systems. The only requirement of the Phoronix Test Suite is PHP. For this demonstration I am using a Fedora 17 virtual machine, however your experience should be fairly similar on whatever Linux distribution you may be using.

The first thing I needed to do was install a few PHP packages. A quick call to yum will take of this for us:
[root@guest1 ~]# yum -y install php-cli php-xml php-gd ...

Next I downloaded and extracted the software and installed it using their script.
[root@guest1 phoronix]# mkdir /tmp/phoronix [root@guest1 phoronix]# cd /tmp/phoronix [root@guest1 phoronix]# [root@guest1 phoronix]# wget --quiet --output-document=phoronix-test-suite-4.2.0.tar.gz [root@guest1 phoronix]# [root@guest1 phoronix]# tar xf phoronix-test-suite-4.2.0.tar.gz [root@guest1 phoronix]# [root@guest1 phoronix]# cd phoronix-test-suite [root@guest1 phoronix-test-suite]# [root@guest1 phoronix-test-suite]# ./install-sh Phoronix Test Suite Installation Completed Executable File: /usr/bin/phoronix-test-suite Documentation: /usr/share/doc/phoronix-test-suite/ Phoronix Test Suite Files: /usr/share/phoronix-test-suite/ [root@guest1 phoronix-test-suite]#

Now that the phoronix software is installed you can run any of the phoronix tests by using the phoronix-test-suite command. You can run the phoronix-test-suite list-suites command to view what test suites are available for you to use. Some of the most notable ones are the disk test suite and the cpu test suites.

NOTE: A license message will appear upon the first run of phoronix-test-suite and will require you to answer a few questions.

In order to view the actual tests within the test suite the phoronix-test-suite info command is useful. For example, to see the tests within the disk test suite you would call phoronix-test-suite info disk as is done below:
[root@guest1 phoronix-test-suite]# phoronix-test-suite info disk Phoronix Test Suite v4.2.0 Disk Test Suite Run Identifier: pts/disk-1.2.1 Suite Version: 1.2.1 Maintainer: Michael Larabel Suite Type: Disk Unique Tests: 13 Suite Description: This test suite is designed to contain real-world disk and file-system tests. pts/disk-1.2.1 * pts/compress-gzip * pts/sqlite * pts/apache * pts/pgbench * pts/compilebench * pts/iozone * pts/dbench * pts/fs-mark * pts/fio * pts/tiobench * pts/postmark * pts/aio-stress * pts/unpack-linux [root@guest1 phoronix-test-suite]#

So how do you run these tests? I like to run my tests in batch mode so that no questions are asked to me during the test execution. To set this up run the phoronix-test-suite batch-setup command in order to answer a few questions that will apply to all of the future batch runs. The answers that I chose are shown in the output below:
[root@guest1 phoronix-test-suite]# phoronix-test-suite batch-setup These are the default configuration options for when running the Phoronix Test Suite in a batch mode (i.e. running phoronix-test-suite batch-benchmark universe). Running in a batch mode is designed to be as autonomous as possible, except for where you'd like any end-user interaction. Save test results when in batch mode (Y/n): Y Open the web browser automatically when in batch mode (y/N): N Auto upload the results to (Y/n): n Prompt for test identifier (Y/n): n Prompt for test description (Y/n): n Prompt for saved results file-name (Y/n): n Run all test options (Y/n): Y Batch settings saved. [root@guest1 phoronix-test-suite]#

Now that we have batch set up we can run a few tests using the phoronix-test-suite batch-benchmark command. I chose to run the pts/iozone and pts/compress-7zip benchmarks.
[root@guest1 phoronix-test-suite]# phoronix-test-suite batch-benchmark pts/iozone pts/compress-7zip ... CLICK TO SEE OUTPUT ...

It should take quite some time for the tests to run. After the tests are complete the results are stored in the ~/.phoronix-test-suite/test-results/ directory. For example, from my test run all files were stored in ~/.phoronix-test-suite/test-results/2012-12-30-1234/. The best way I have found to view the test results is to fire up a web browser and point it to the index.html file within the test results directory:
[root@guest1 phoronix-test-suite]# firefox ~/.phoronix-test-suite/test-results/2012-12-30-1234/index.html

An example of the test results output for the test I ran can be found HERE

The test results for a single test run are nice, but the graphs really help when comparing runs from multiple tests. i.e you tweak some setting and then run the benchmarks again to see the performance impact. I performed two runs of the compress-7zip benchmark to do just this. The two test runs were placed into the 2012-12-30-2102 and 2012-12-30-2106 directories within the test-results directory. In order to compare the outputs side by side the phoronix-test-suite merge-results command is used.

[root@guest1 phoronix-test-suite]# phoronix-test-suite merge-results 2012-12-30-2102 2012-12-30-2106 Merged Results Saved To: /root/.phoronix-test-suite/test-results/merge-1844/composite.xml Do you want to view the results in your web browser (y/N): N [root@guest1 phoronix-test-suite]#

For these test runs I didn't tweak any settings on the system so they are pretty much the same result, but take a look at the merged test results HERE to get an idea of what I mean about the side-by-side comparison.

Take an opportunity to check out the phoronix documentation or the phoronix-test-suite man page to discover more features!

Happy Benchmarking!

Dusty Mabe

Trace Function Calls Using GDB Revisited!

In an earlier post I discussed how to trace calls using GDB so that function calls and their arguments can easily be viewed. What I neglected to mention was rbreak , a feature of GDB to be able to set breakpoints using a regular expression.

Using rbreak you can get the same functionality but with much less effort. For example, to get the same behavior as before (setting a breakpoint on each function call and printing a trace of the bottom most level) all you need to provide to GDB are the following commands:

set args 4 10 rbreak file.c:. command silent backtrace 1 continue end run

I have placed these commands in a file called gdb_commands. The rbreak command above tells GDB to set a breakpoint on any function call within file.c. With a quick call to GDB we find almost the exact same functionality as we had before.

dustymabe@laptop: gdbpost>gdb -quiet -command=gdb_commands ./a.out Reading symbols from /content/gdbpost/a.out...done. Breakpoint 1 at 0x400578: file file.c, line 12. int main(int, char **); Breakpoint 2 at 0x400546: file file.c, line 7. int modulo(int, int); Breakpoint 3 at 0x40055f: file file.c, line 8. int plus(int, int); Breakpoint 4 at 0x400533: file file.c, line 6. int squared(int); #0 main (argc=3, argv=0x7fffffffe698) at file.c:12 #0 squared (x=4) at file.c:6 #0 squared (x=10) at file.c:6 #0 modulo (x=100, y=16) at file.c:7 #0 plus (x=100, y=100) at file.c:8 #0 modulo (x=200, y=16) at file.c:7 #0 plus (x=200, y=100) at file.c:8 #0 modulo (x=300, y=16) at file.c:7 #0 plus (x=300, y=100) at file.c:8 #0 modulo (x=400, y=16) at file.c:7 LCM is 400 [Inferior 1 (process 1579) exited with code 013] Missing separate debuginfos, use: debuginfo-install glibc-2.15-56.fc17.x86_64 (gdb) quit

In my previous example, if you had 50 functions that you wanted to trace, you needed 50 break commands as well as 50 command blocks.

Considering this fact, it is apparent that rbreak is much more efficient and friendly for when you are setting breakpoints from the GDB cli. Another extremely useful feature is that it also works for c++ classes, so you can break on any function in a class regardless of what file the function is defined in.

To show an example of this I converted all the math function calls from the program in file.c to a c++ class. I put the result in a file called (shown below).

#include <stdio.h> #include <stdlib.h> // A program that will square two integers and then find the LCM // of the resulting two integers. class Math { public: int squared(int x) { return x*x; } int modulo(int x, int y) { return x%y; } int plus(int x, int y) { return x+y; } }; int main(int argc, char *argv[]) { int x, y, x2, y2, tmp; if (argc != 3) return 0; Math* m = new Math; x = atoi(argv[1]); y = atoi(argv[2]); x2 = m->squared(x); y2 = tmp = m->squared(y); while (1) { if (m->modulo(tmp,x2) == 0) break; tmp = m->plus(tmp,y2); } printf("LCM is %dn", tmp); }

I then used the rbreak Math:: command in the following GDB commands file (I named this one gdb_commands2) to run GDB and set up the break points.

set args 4 10 rbreak Math:: command silent backtrace 1 continue end run

And.. Here we are with the final result:

dustymabe@laptop: gdbpost>gdb -quiet -command=gdb_commands2 ./a.out Reading symbols from /content/gdbpost/a.out...done. Breakpoint 1 at 0x400758: file, line 9. int Math::modulo(int, int); Breakpoint 2 at 0x400776: file, line 10. int Math::plus(int, int); Breakpoint 3 at 0x400741: file, line 8. int Math::squared(int); #0 Math::squared (this=0x601010, x=4) at #0 Math::squared (this=0x601010, x=10) at #0 Math::modulo (this=0x601010, x=100, y=16) at #0 Math::plus (this=0x601010, x=100, y=100) at #0 Math::modulo (this=0x601010, x=200, y=16) at #0 Math::plus (this=0x601010, x=200, y=100) at #0 Math::modulo (this=0x601010, x=300, y=16) at #0 Math::plus (this=0x601010, x=300, y=100) at #0 Math::modulo (this=0x601010, x=400, y=16) at LCM is 400 [Inferior 1 (process 2304) exited normally] Missing separate debuginfos, use: debuginfo-install glibc-2.15-56.fc17.x86_64 libgcc-4.7.0-5.fc17.x86_64 libstdc++-4.7.0-5.fc17.x86_64 (gdb) quit



PS - If you are new to GDB I have found a great reference for beginners here .

Mount Complex Disk Images Using libguestfs

In my previous post I went over two ways to mount a partition within a disk image file. There is actually another easy way to do this by utilizing some of the tools that the virt community has provided us in recent years.

libguestfs is a fairly comprehensive library for manipulating guest disk images and filesystems. It turns out that the tools work pretty well even for disk images that aren't specific to any virtualized guest; after all a disk image is a disk image.

The guestmount utility is provided as part of the libguestfs-tools-c package in Fedora 17 and allows us the ability to (in)directly mount the second partition of our disk image. This is shown below:

dustymabe@media: > guestmount -a /mnt/lenovo.img -m /dev/sda2 --ro /tmp/mnt dustymabe@media: > ls /tmp/mnt/Users/ All Users Default Default User desktop.ini dustymabe Public

The only drawback I can see at this point from using this tool is that it is using fuse under the covers to mount the filesystem. This has some advantages and disadvantages but typically results in lower performance. Nevertheless, guestmount and the other tools provided by libguestfs are no doubt becoming very useful for manipulating disk images.


Dusty Mabe

Mounting a Partition Within a Disk Image

Last time I walked through creating a sparse disk image using dd and cp --sparse=always. OK, we have a disk image. Now what?

Normally it would suffice to just set up a loop device and then mount, but this disk image doesn't just contain a filesystem. It has 4 partitions each with their own filesystem. This means in order to mount one of the filesystems we have to take a few extra steps.

There is an easy and a hard way to do this. I'll start with the hard way..

The Hard Way

To manually mount a particular partition on the disk we need to find a little information about where the partition is located on the disk. We need to use fdisk to find the offsets of the start and end of the partition.

dustymabe@media: mnt>sudo fdisk -l /mnt/lenovo.img Disk /mnt/lenovo.img: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xf82e2761 Device Boot Start End Blocks Id System /mnt/lenovo.img1 * 2048 411647 204800 7 HPFS/NTFS/exFAT /mnt/lenovo.img2 411648 884611071 442099712 7 HPFS/NTFS/exFAT /mnt/lenovo.img3 884611072 946051071 30720000 7 HPFS/NTFS/exFAT /mnt/lenovo.img4 946051072 976361471 15155200 12 Compaq diagnostics

From the fdisk output we can see that the second partition starts at 411648 sectors times 512 bytes (210763776 bytes) into the disk image. We can also see that the partition ends at 884611071 sectors times 512 bytes (452920868352 bytes). This means that the size of the partition is 452920868352 - 210763776 = 452710104576 bytes.

We can use this information to set up a loop device using losetup like so:

dustymabe@media: >sudo losetup -v -f -o 210763776 --sizelimit 452710104576 /mnt/lenovo.img dustymabe@media: > dustymabe@media: >losetup -a /dev/loop0: []: (/mnt/lenovo.img), offset 210763776, sizelimit 452710104576 dustymabe@media: > dustymabe@media: >sudo blkid /dev/loop0 /dev/loop0: LABEL="Windows7_OS" UUID="7C64B5C764B58504" TYPE="ntfs"

Finally, the loop device can be mounted just like a normal block device:

dustymabe@media: >sudo mount -o ro /dev/loop0 ~/Desktop/mount/ dustymabe@media: >ls ~/Desktop/mount/Users All Users Default Default User desktop.ini dustymabe Public

The Easy Way

You can use partx to probe a disk image and detect all partitions. You can also have partx tell the kernel about all partitions and add devices for each partition. An example of this is shown below:

dustymabe@media: >sudo losetup -v -f /mnt/lenovo.img dustymabe@media: >sudo losetup -a /dev/loop0: [2145]:12 (/mnt/lenovo.img) dustymabe@media: > dustymabe@media: >sudo partx --show /dev/loop0 [sudo] password for dustymabe: NR START END SECTORS SIZE NAME UUID 1 2048 411647 409600 200M 2 411648 884611071 884199424 421.6G 3 884611072 946051071 61440000 29.3G 4 946051072 976361471 30310400 14.5G dustymabe@media: > dustymabe@media: >sudo partx -v --add /dev/loop0 partition: none, disk: /dev/loop0, lower: 0, upper: 0 /dev/loop0: partition table type 'dos' detected /dev/loop0: partition #1 added /dev/loop0: partition #2 added /dev/loop0: partition #3 added /dev/loop0: partition #4 added dustymabe@media: > dustymabe@media: > dustymabe@media: >sudo blkid /dev/loop0* /dev/loop0p1: LABEL="SYSTEM_DRV" UUID="0840B6DE40B6D224" TYPE="ntfs" /dev/loop0p2: LABEL="Windows7_OS" UUID="7C64B5C764B58504" TYPE="ntfs" /dev/loop0p3: LABEL="LENOVO" UUID="7652C00A52BFCCDD" TYPE="ntfs" /dev/loop0p4: LABEL="LENOVO_PART" UUID="983479C03479A244" TYPE="ntfs"

Now you can directly mount /dev/loop0p2 just as we did before.

dustymabe@media: >sudo mount -o ro /dev/loop0p2 ~/Desktop/mount/ dustymabe@media: >ls ~/Desktop/mount/Users/ All Users Default Default User desktop.ini dustymabe Public dustymabe@media: >

There ya go.. Two ways to mount partitions within a disk image. If I get a chance I'll go over a third using some free tools from the virt community.