The Ultimate Guide to .NET Performance Testing with BenchmarkDotNet
Pramesh Kc.
Posted on November 30, 2024
Introduction
Have you ever wondered if your code could run faster? Or maybe you're choosing between different ways to write something and aren't sure which is better? That's where benchmarking comes in! It's like a stopwatch for your code that helps you make smart decisions about how to write your programs.
Why BenchmarkDotNet?
BenchmarkDotNet stands out for several reasons:
- Statistical analysis to ensure reliable results
- Cross-platform support (.NET Core, .NET Framework, Mono)
- Hardware intrinsics reporting
- Easy-to-read reports in various formats (JSON, HTML, CSV)
- Memory allocation and garbage collection statistics
Your First Steps into Benchmarking
Step 1: Setting Up Your Project
First, create a new console application:
dotnet new console -n MyFirstBenchmark
cd MyFirstBenchmark
Now, add the BenchmarkDotNet package:
dotnet add package BenchmarkDotNet
Step 2: Your First Benchmark
Let's create a simple benchmark that compares different ways to join strings - something every developer does! Here's a beginner-friendly example:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
public class StringBenchmarks
{
// This is our test data
private string firstName = "John";
private string lastName = "Smith";
[Benchmark]
public string UsingPlusOperator()
{
return "Hello, " + firstName + " " + lastName + "!";
}
[Benchmark]
public string UsingStringInterpolation()
{
return $"Hello, {firstName} {lastName}!";
}
// This is where we run our benchmarks
public static void Main(string[] args)
{
BenchmarkRunner.Run<StringBenchmarks>();
}
}
Step 3: Running Your Benchmark
- Open your terminal
- Navigate to your project folder
- Run this command:
dotnet run -c Release
The -c Release
part is important! Always run benchmarks in Release mode for accurate results.
Understanding Your Results
When your benchmark finishes, you'll see a table like this:
| Method | Mean | Error |
|-------------------- |---------:|---------:|
|UsingPlusOperator | 45.23 ns | 0.85 ns |
|UsingInterpolation | 43.12 ns | 0.81 ns |
Let's break down what these numbers mean:
- Mean: The average time it took to run (in nanoseconds here)
- Error: How much the results might vary
- ns means nanoseconds (that's really, really fast!)
Memory Diagnostics
To measure memory allocations:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
[MemoryDiagnoser]
public class StringBenchmarks
{
private const string name = "John";
[Benchmark(Baseline = true)]
public string UsingPlus()
{
return "Hello, " + name + "!";
}
[Benchmark]
public string UsingInterpolation()
{
return $"Hello, {name}!";
}
public static void Main(string[] args)
{
BenchmarkRunner.Run<StringBenchmarks>();
}
}
Understanding Results
Your output will look like this:
| Method | Mean | Error | StdDev | Allocated |
|------------------|----------|----------|-----------|------------|
| UsingPlus | 12.23 ns | 0.123 ns | 0.115 ns | 24 B |
| UsingInterpolation| 11.45 ns | 0.118 ns | 0.110 ns | 24 B |
Key metrics explained:
- Mean: Average time per operation
- Error: Statistical error margin
- StdDev: How much results vary
- Allocated: Memory used
Collection Performance
Compare different collection types:
[MemoryDiagnoser]
public class CollectionBenchmarks
{
[Params(100, 10000)]
public int Size { get; set; }
[Benchmark]
public void List()
{
var list = new List<int>(Size);
for (int i = 0; i < Size; i++)
list.Add(i);
}
[Benchmark]
public void Array()
{
var array = new int[Size];
for (int i = 0; i < Size; i++)
array[i] = i;
}
}
Async Operations
Benchmarking async code requires special attention:
public class AsyncBenchmarks
{
private HttpClient client = new();
[Benchmark]
public async Task<string> AsyncDownload()
{
return await client.GetStringAsync("http://example.com");
}
[Benchmark]
public async Task<string> AsyncDownloadWithTimeout()
{
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
return await client.GetStringAsync("http://example.com", cts.Token);
}
}
Common Mistakes to Avoid
- ❌ Don't run benchmarks in Debug mode
- ❌ Don't benchmark with tiny amounts of work
- ❌ Don't ignore the warm-up phase
Tips for Success
- ✅ Always use Release mode
- ✅ Test with realistic data
- ✅ Run benchmarks multiple times
- ✅ Keep your test cases simple at first
Next Steps
Once you're comfortable with these basics, you can try:
- Testing different sizes of data
- Comparing different .NET versions
- Measuring memory usage
- Benchmarking your own code
Remember:
- Start with simple comparisons
- Always run in Release mode
- Use realistic data
- Take your time understanding the results
You don't need to be an expert to start benchmarking! Begin with these simple examples, and as you get more comfortable, you can explore more advanced features.
Happy coding! 🎉
Posted on November 30, 2024
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
November 30, 2024