How to store app secrets for your ASP .NET Core project

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

This article is for you that is either completely new to ASP .NET Core or is currently storing your secrets in config files that you may or may not check in by mistake. Keep secrets separate, store them using the Secret management tool in dev mode, and look into services like Azure KeyVault for production.

Let's talk about app secrets and configuration and why we need a tool to manage it. There are a few reasons why this is something that needs to be managed and preferably by a tool:

Separate config/secrets from source code, your configuration is sensitive, configuration strings may contain passwords or API keys or other secrets. Having this information exposed may leave your system vulnerable. You want to avoid storing any of the data in source code as your source code will most likely end up in a repo on GitHub or a similar place. Even though it's a private repo it may be exposed. Better to store this elsewhere.
Values are different in different environments, the values you use for DEV, Staging, and Prod are hopefully different when it comes to connecting to a database or API. You need to acknowledge what's different so you can separate this out as configuration that needs to be replaced per environment
OSs are different, you might think that it's enough to make all secrets into environment variables, and you are done. However, you might have so much configuration that you need to organize it in a hierarchy like so api:<apitype>:<apikey>. One problem though, : isn't supported on all OSs but other characters are like ___. The point being is that you want an abstraction layer to organize your secrets/config.

References

Secret manager tool

When you install .NET Core you get a built-in tool to help you managing configuration and secrets. It addresses a lot of the concerns that we covered in the last section. However, there are some things you should know before we continue:

The tool is great for local dev, The secret manager tool is great for local development but that's where it should stay.
Environment variables are not safe, your machine might be compromised and Environment Variables are plain text and not encrypted. So even though it's tempting to rely on Environment Variables and store those in AppSetting in Azure you want to look into more safe ways of handling secrets like KeyVault.

The secret manager tool is a command-line tool that stores your secrets in a JSON file. Once you initialize the tool for a specific project it generates a Secrets Id and creates a JSON file in a place that's OS-dependent:

For MAC

~/.microsoft/usersecrets/<user_secrets_id>/secrets.json

For Windows

%APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json

The idea is that you initialize in the root of an ASP .NET Core project and the Secrets Id is placed in the project file. It then works with .NET Core and some provider code to make it easy to retrieve and store secrets through code.

DEMO manage secrets

Let's try to cover the following areas:

Initialization, this is how you generate an id and creates a file that will contain your secrets
Setting a value, this is about setting a value, either from a terminal or from code.
Accessing a value, this can be done from both terminal and code.
Removing a value, it's good to know how to remove a value when you no longer need it. It can be removed from both terminal and code.

Initialization

First, create a .NET Core Web API project by typing the following command in the terminal:

   dotnet new webapi -o api --no-https
   cd api

Type the following command

   dotnet user-secrets init

The terminal should give you an output like so:

   Set UserSecretsId to '<secret-id>' for MSBuild project '/path/to/your/project/project.csproj'.

Open up your project file that ends in .csproj. Locate an entry under PropertyGroup looking like so:

   <UserSecretsId>secret-id</UserSecretsId>

This UserSecretsId is how the secrets JSON file is connected to your app.

Setting a value

Before we create a secret let's learn how to list the secrets we currently have (there should be none at this point). Type the following command in the terminal:

dotnet user-secrets list

You should get the following output:

No secrets configured for this application.

Next, let's create a secret.

In the terminal type the following:

   dotnet user-secrets set "ApiKey" "12345"

List the content of the secret store again:

   dotnet user-secrets list

Now you get the following output:

   ApiKey = 12345

You can also create a namespace with secrets for when you want to group things that go together like:

ProductsUrl
ProductsApiKey

Type the following command to create grouped secrets:

   dotnet user-secrets set "Products:Url" "http://path/to/product/url"
   dotnet user-secrets set "Products:ApiKey" "123abc"

You should get the following output:

   Successfully saved Products:Url = http://path/to/product/url to the secret store.
   Successfully saved Products:ApiKey = 123abc to the secret store.

List your secrets content:

   Products:Url = http://path/to/product/url
   Products:ApiKey = 123abc
   ApiKey = 12345

The above might look exactly like when you stored ApiKey but there is a difference when accessing. Let's try to access next.

Accessing a value

The Configuration API will help us retrieve our secrets in source code. It's a powerful API that is capable of reading data from various sources like appsettings.json, environment variables, KeyVault, Command-line, and much more, with the help of dedicated providers that can be added at startup. It's worth stressing this API helps us only in development mode when it comes to reading secrets. The secret management tool is only meant for development so that works for us.

Open up Startup.cs and note how the constructor already inject it like so:

   public Startup(IConfiguration configuration) {}

Locate ConfigureServices() method and add the following code to retrieve and display the secret:

   public void ConfigureServices(IServiceCollection services)
   {
      ApiKey = Configuration["Products:Url"];
      Console.WriteLine(ApiKey);
      services.AddControllers();
   }

Build and run your project by running the following commands:

   dotnet build && dotnet run

You should get the following output at the top:

   http://path/to/product/url

Great our secret is listed where it should be. What if we want to access these values from somewhere else other than Startup.cs, like from a controller or a service? Yea we can do that, by using the built-in dependency injection.

We are about to register a singleton, this is something we can use and inject it into a service or controller. The singleton will contain the API keys or other secrets we might need to access.

Create a file AppConfiguration.cs and give it the following content:

  namespace webapi_secret
  {
    public class AppConfiguration
    {
      public string ApiKey { get; set; }
      public Product Product { get; set; }
    }
  }

Go back to Startup.cs, locate the ConfigureServices() method and add the following lines:

  var config = new AppConfiguration();
  config.ApiKey = Configuration["Products:Url"];
  services.AddSingleton<AppConfiguration>(config);

Now we have a singleton we can use anywhere :)

Create a ProductsController.cs and ensure it looks like so:

  using Microsoft.AspNetCore.Mvc;

  namespace webapi_secret.Controllers
  {
    [ApiController]
    [Route("[controller]")]
    public class ProductsController : ControllerBase
    {
      AppConfiguration _config;
      public ProductsController(AppConfiguration config)
      {
        this._config = config;
      }

      [HttpGet]
      public string Get()
      {
        return this._config.ApiKey;
      }
    }
  }

Note how we inject AppConfiguration in the constructor and assign the instance this._config = config;. Note also how we construct a method Get() and return config key:

  [HttpGet]
  public string Get()
  {
    return this._config.ApiKey;
  }

Build and run this with the following command:
```
dotnet build && dotnet run
```
Navigate to https://localhost:5001/products

Note, you can do it like this and have a configuration singleton that you use where you need it or you can create your services and register them to the DI container with the config passed through the constructor, like so:

Create a file HttpService.cs and give it the following content:

   namespace webapi_secret
    {
      public class HttpService
      {
        private string _apiKey;
        public HttpService(string apiKey)
        {
          this._apiKey = apiKey;
        }

        public string ApiKey { get { return _apiKey; } }
      }
    }

Go to the file Startup.cs and locate the ConfigureService() method and the following line:

   services.AddSingleton<HttpService>(new HttpService(Configuration["ApiKey"]));

Now you can inject HttpService wherever you need it and know that it is configured with an API key.

Whether you want a configuration instance that you inject or if you want services created with the necessary keys is up to you.

Wait, what those keys we created Products::ApiKey, all that talk of a namespace?

Yea we can deal with those in a very elegant way.

Create a file ProductConfiguration.cs and give it the following content:

   public class ProductConfiguration
   {
     public string Url { get; set; }
     public string ApiKey { get; set; }
   }

Go to Startup.cs and add the following code to ConfigureServices():

   var productConfig = Configuration.GetSection("Products")
                          .Get<ProductConfiguration>();
   services.AddSingleton<ProductConfiguration>(productConfig);

The GetSection() method allows us to grab a namespace and then map everything on that namespace and map it into a type. This is great now we can have dedicated parts of our secrets mapped to dedicated configuration classes

Just like with AppConfiguration we can inject this where we please. Update ProductsController.cs to look like so:

   using Microsoft.AspNetCore.Mvc;

    namespace webapi_secret.Controllers
    {
      [ApiController]
      [Route("[controller]")]
      public class ProductsController : ControllerBase 
      {
        AppConfiguration _config;
        ProductConfiguration _productConfig;
        public ProductsController(AppConfiguration config, ProductConfiguration productConfiguration)
        {
          this._config = config;
          this._productConfig = productConfiguration;
        }

        [HttpGet]
        public string Get()
        {
          // return this._config.ApiKey;
          return this._productConfig.ApiKey;
        }
      }
    }

Removing a value

Lastly, how do we clean up? Use the command remove and give it the name of the key, like so:

dotnet user-secrets remove "ApiKey"

There's also a clear command that removes all keys, be careful with that one though:

dotnet user-secret clear

Summary

We discussed why it's a bad idea to have secrets in your source code, i.e you can check it in by mistake. Additionally, we talked about how the secret manager tool can help you while developing to keep track of your secrets. Then we showed how to manage secrets and thus covering:

Adding secrets
Reading secrets from command-line and from code
Configure DI instances to be populated by secrets
Remove secrets

I hope this was helpful.

Blog