Azure AppService HttpClientFactory - use cached but fresh http message handlers.

There is one trend in the technical industry: recommendations are intended to be elevated to requirements. Times when developers read documentation from cover to cover are long gone, and we have to react to issues that a platform enforces. Migration from Service Fabric clusters running on Virtual Machine Scale Set to Azure App Service enforces stricter requirements for .NET outbound connection practices. In 2015, the first generation of App Service was running on dedicated VMs. Modern Azure App Service (2020+) (WebSites and AzureFunction) work environments have a 128 SNAT ports limit. MSDN articles are sometimes controversial. It took reading many of them, practicing coding, reading source code, and debugging DefaultHttpClientFactory internals to figure out a universal coding practice:

1. Use cached but fresh http message handlers
   1. Do not create HttpClient using its constructor because it creates many HttpMessageHandlers and reaches SNAT ports limit.
   2. Don’t use HttpClient singleton because it makes linked HttpMessageHandlers not fresh and may cause DNS problems.
2. Use AddHttpClient that registers IHttpClientFactory and IHttpMessageHandlerFactory singletons dependency injection that are implemented in DefaultHttpClientFactory. You may inject these factories into your own services with any DI lifetimes (transient or singleton). DefaultHttpClientFactory internally contains dictionary of named HttpMessageHandler’s instances that are expired in 2 minutes after creation (to avoid DNS issues).
3. Use clients only in a transient way (for operations that do not take longer than a few minutes).
   1. IHttpClientFactory.CreateClient() to create HttpClient and use it only with a transient lifecycle. HttpClient contains a hard link to HttpMessageHandler. You may dispose HttpClient created by IHttpClientFactory, but that will not dispose the referenced HttpMessageHandler. Typed HttpClient is also a transient-only approach. DefaultHttpClientFactory caches HttpMessageHandler after IHttpClientFactory.CreateClient() invocation for 2 minutes. Transient usage of HttpClient will guarantee that cached but fresh handlers will be used.
   2. Use IHttpMessageHandlerFactory.CreateHandler() to create HttpMessageHandler that may be used for transient connection wrappers (e.g. DocumentClient, KeyVaultClient).
4. If you really need to get out of the SNAT ports limit on App Service, then consider a few options.
   1. Use App Service Environments
   2. Use NAT Gateways, service endpoints, private endpoints to avoid SNAT restrictions.
   3. Use other protocol connection pools (e.g. SQL connection pools).

Here is the list of articles and key concepts that it is possible to mine from them:

• DI scopes in IHttpClientFactory message handlers don’t work like you think they do This is a good deep article that proves the results of our debugging.
• Use IHttpClientFactory to implement resilient HTTP requests Provides general guidance to use IHttpClientFactory with dependency injection.
• Make HTTP requests using IHttpClientFactory in ASP.NET Core Provides
• Troubleshooting intermittent outbound connection errors in Azure App Service Contains symptoms, causes, and general recommendations to avoid problems.
• Manage connections in Azure Functions provides wrong recommendation to use singleton for HttpClient and CosmosClient because it may cause DNS issue. Azure Functions support Dependency Injection and IHttpClientFactory can be injected.
• Improper Instantiation antipattern another MSDN article that recommends using a singleton that fixes the SNAT ports issue but creates DNS issues.