The Ultimate Guide to .NET Performance Testing with BenchmarkDotNet

prameshkc

Pramesh Kc.

Posted on November 30, 2024

The Ultimate Guide to .NET Performance Testing with BenchmarkDotNet

Introduction

Have you ever wondered if your code could run faster? Or maybe you're choosing between different ways to write something and aren't sure which is better? That's where benchmarking comes in! It's like a stopwatch for your code that helps you make smart decisions about how to write your programs.

Why BenchmarkDotNet?

BenchmarkDotNet stands out for several reasons:

  • Statistical analysis to ensure reliable results
  • Cross-platform support (.NET Core, .NET Framework, Mono)
  • Hardware intrinsics reporting
  • Easy-to-read reports in various formats (JSON, HTML, CSV)
  • Memory allocation and garbage collection statistics

Your First Steps into Benchmarking

Step 1: Setting Up Your Project

First, create a new console application:

dotnet new console -n MyFirstBenchmark
cd MyFirstBenchmark
Enter fullscreen mode Exit fullscreen mode

Now, add the BenchmarkDotNet package:

dotnet add package BenchmarkDotNet
Enter fullscreen mode Exit fullscreen mode

Step 2: Your First Benchmark

Let's create a simple benchmark that compares different ways to join strings - something every developer does! Here's a beginner-friendly example:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

public class StringBenchmarks
{
    // This is our test data
    private string firstName = "John";
    private string lastName = "Smith";

    [Benchmark]
    public string UsingPlusOperator()
    {
        return "Hello, " + firstName + " " + lastName + "!";
    }

    [Benchmark]
    public string UsingStringInterpolation()
    {
        return $"Hello, {firstName} {lastName}!";
    }

    // This is where we run our benchmarks
    public static void Main(string[] args)
    {
        BenchmarkRunner.Run<StringBenchmarks>();
    }
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Running Your Benchmark

  1. Open your terminal
  2. Navigate to your project folder
  3. Run this command:
dotnet run -c Release
Enter fullscreen mode Exit fullscreen mode

The -c Release part is important! Always run benchmarks in Release mode for accurate results.

Understanding Your Results

When your benchmark finishes, you'll see a table like this:

|               Method |     Mean |    Error |
|-------------------- |---------:|---------:|
|UsingPlusOperator    | 45.23 ns | 0.85 ns  |
|UsingInterpolation   | 43.12 ns | 0.81 ns  |
Enter fullscreen mode Exit fullscreen mode

Let's break down what these numbers mean:

  • Mean: The average time it took to run (in nanoseconds here)
  • Error: How much the results might vary
  • ns means nanoseconds (that's really, really fast!)

Memory Diagnostics

To measure memory allocations:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]
public class StringBenchmarks
{
    private const string name = "John";

    [Benchmark(Baseline = true)]
    public string UsingPlus()
    {
        return "Hello, " + name + "!";
    }

    [Benchmark]
    public string UsingInterpolation()
    {
        return $"Hello, {name}!";
    }

    public static void Main(string[] args)
    {
        BenchmarkRunner.Run<StringBenchmarks>();
    }
}
Enter fullscreen mode Exit fullscreen mode

Understanding Results

Your output will look like this:

|            Method |     Mean |    Error |   StdDev |  Allocated |
|------------------|----------|----------|-----------|------------|
|        UsingPlus | 12.23 ns | 0.123 ns | 0.115 ns |      24 B |
| UsingInterpolation| 11.45 ns | 0.118 ns | 0.110 ns |      24 B |
Enter fullscreen mode Exit fullscreen mode

Key metrics explained:

  • Mean: Average time per operation
  • Error: Statistical error margin
  • StdDev: How much results vary
  • Allocated: Memory used

Collection Performance

Compare different collection types:

[MemoryDiagnoser]
public class CollectionBenchmarks
{
    [Params(100, 10000)]
    public int Size { get; set; }

    [Benchmark]
    public void List()
    {
        var list = new List<int>(Size);
        for (int i = 0; i < Size; i++)
            list.Add(i);
    }

    [Benchmark]
    public void Array()
    {
        var array = new int[Size];
        for (int i = 0; i < Size; i++)
            array[i] = i;
    }
}
Enter fullscreen mode Exit fullscreen mode

Async Operations

Benchmarking async code requires special attention:

public class AsyncBenchmarks
{
    private HttpClient client = new();

    [Benchmark]
    public async Task<string> AsyncDownload()
    {
        return await client.GetStringAsync("http://example.com");
    }

    [Benchmark]
    public async Task<string> AsyncDownloadWithTimeout()
    {
        using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
        return await client.GetStringAsync("http://example.com", cts.Token);
    }
}
Enter fullscreen mode Exit fullscreen mode

Common Mistakes to Avoid

  1. ❌ Don't run benchmarks in Debug mode
  2. ❌ Don't benchmark with tiny amounts of work
  3. ❌ Don't ignore the warm-up phase

Tips for Success

  1. ✅ Always use Release mode
  2. ✅ Test with realistic data
  3. ✅ Run benchmarks multiple times
  4. ✅ Keep your test cases simple at first

Next Steps

Once you're comfortable with these basics, you can try:

  1. Testing different sizes of data
  2. Comparing different .NET versions
  3. Measuring memory usage
  4. Benchmarking your own code

Remember:

  • Start with simple comparisons
  • Always run in Release mode
  • Use realistic data
  • Take your time understanding the results

You don't need to be an expert to start benchmarking! Begin with these simple examples, and as you get more comfortable, you can explore more advanced features.

Happy coding! 🎉

💖 💪 🙅 🚩
prameshkc
Pramesh Kc.

Posted on November 30, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related