Preface:
In previous articlesImplementation of response compression in CoreIt is mentioned that the main job of the server is based onContent-EncodingThe header information is determined in which way to compress and return. Someone in the group asked before, is it necessary to compress the requests on the server with such high network bandwidth now? Indeed, nowadays, distributed and load balancing technologies are so mature, many scenarios that require processing high-concurrency big data can be performed by adding server nodes. However, when resources are limited, or when there is no need to add new server nodes for a certain point, we still need to use some conventional processing methods of the program itself to handle it. The author personally believes that the use scenario of response compression is like this. When the bandwidth pressure is relatively tight and the CPU resources are sufficient, the overall effect of using response compression is still relatively obvious.
There is compression and decompression, and the decompression is to request the client to handle it. For example, browsers are our most commonly used Http client. Many browsers default to when we make requests (such as when we browse web pages).Request HeadAdded inContent-Encoding, and then process the relevant decompression according to the response information. All of this is because the browser already has built-in mechanisms about requesting compression and decompression. There are many similar ones, such as the commonly used proxy packet capture toolFilderThis mechanism is also built-in. It just needs to be processed manually, but the implementation method is the same. Sometimes we also need to use this mechanism in the process of writing programs ourselves, in the traditional.Net HttpWebRequestThere is no such mechanism in the class library, and later it was added to the version.HttpClient, There is a built-in mechanism to handle this operation.
1. How to use
First, let's take a look at it directlyHttpClientHow to deal with response compression
//Customize HttpClientHandler instanceHttpClientHandler httpClientHandler = new HttpClientHandler
{
AutomaticDecompression =
};
//Use the constructor that passes a custom HttpClientHandler instanceusing (HttpClient client = new HttpClient(httpClientHandler))
{
var response = await ($"http://MyDemo/Home/GetPerson?userId={userId}");
}
This operation is very simple, we are not operatingHttpClientbutHttpClientHandlerThe properties in our previous articles[.NET Core HttpClientIt has been discussed in Source Code Exploration],HttpClientThe essence is actuallyHttpMessageHandler,andHttpClientWhat is really used isHttpMessageHandlerThe most important subclassHttpClientHandler, all request operations are passedHttpMessageHandlerProgress. We can seeAutomaticDecompressionWhat you accept isDecompressionMethodsEnumeration, since it is an enum, it means that it contains more than one value. Let's check it out next.DecompressionMethodsSource code in
[Flags]
public enum DecompressionMethods
{
// Use all compression decompression algorithms. All = -1,
// No decompression None = 0x0,
// Use gzip decompression algorithm GZip = 0x1,
// Use deflate decompression algorithm Deflate = 0x2,
// Use Brotli decompression algorithm Brotli = 0x4
}
This enumeration is based on common output decompression algorithms by default. Let's take a look at it.HttpClientFactoryHow to deal with response compression. In previous articles [.NETCore HttpClientFactory+ConsulWe have discussed in Implementing Service Discovery]HttpClientFactoryThe roughly working method is defaultPrimaryHandlerThe HttpClientHandler instance is passed, and it is registered with usHttpClientFactoryIt can be passed whenConfigurePrimaryHttpMessageHandlerCustomizePrimaryHandlerThe default value, next we implement the specific code
("mydemo", c =>
{
= new Uri("http://MyDemo/");
}).ConfigurePrimaryHttpMessageHandler(provider=> new HttpClientHandler
{
AutomaticDecompression =
});
Actually registeringHttpClientFactoryYou can also use customHttpClientThe specific usage method is like this
("mydemo", c =>
{
= new Uri("http://MyDemo/");
}).ConfigureHttpClient(provider => new HttpClient(new HttpClientHandler
{
AutomaticDecompression =
}));
HttpClientIt has indeed helped us do a lot of things, and we only need to simply configure it to enable the processing of response compression. This has aroused us even moreHttpClientNext, we will check how it initiates a compressible request and decompresses the response result through the source code.
2. Source code exploration
Through the above usage, we know that no matter which form is used, it will ultimately be targeted.HttpClientHandlerDo configuration operations, let's check it outHttpClientHandlerIn classAutomaticDecompressionThe code of the attribute
public DecompressionMethods AutomaticDecompression
{
get => _underlyingHandler.AutomaticDecompression;
set => _underlyingHandler.AutomaticDecompression = value;
}
Its own value operation comes from_underlyingHandlerThis object, that is, both read and set are in operation_underlyingHandler.AutomaticDecompression, we found_underlyingHandlerThe declared location of the object
private readonly SocketsHttpHandler _underlyingHandler;
Let me explain it here,HttpClientThe substantive work category isHttpClientHandler,andHttpClientHandlerThe real request is based onSocketsHttpHandlerThis class, that isSocketsHttpHandleris the original class that initiates the request.HttpClientHandlerIn essence, it is throughSocketsHttpHandlerThe Http request initiated, let's check it nextSocketsHttpHandlerHow are classes handledAutomaticDecompressionThis property
public DecompressionMethods AutomaticDecompression
{
get => _settings._automaticDecompression;
set
{
CheckDisposedOrStarted();
_settings._automaticDecompression = value;
}
}
Here_settingsIt is no longer a specific functional class, but is used for initialization or storageSocketsHttpHandlerConfiguration class for some attribute values
private readonly HttpConnectionSettings _settings = new HttpConnectionSettings();
We are not analyzing hereSocketsHttpHandlerOut other codes other than processing response compression, so I won't look at these in detail, just look up_settings._automaticDecompressionThe attribute reference is where the code is finally found
if (settings._automaticDecompression != )
{
handler = new DecompressionHandler(settings._automaticDecompression, handler);
}
It's clearer here. The real process of request response compression is all related toDecompressionHandlermiddle. As we said before,HttpClientThe real way to work is to implement itHttpMessageHandlerThe subclass of the work is working, and it encapsulates the implementation modules of different functions into specificHandlermiddle. When you need to use the function of which module, use the corresponding module directlyHandlerJust send a processing request by the operation class. This design idea is CoreIt is also vividly reflected in it. CoreIt uses different endpoints to process and output requests. Through these we can knowDecompressionHandlerThis is the theme today, let's check it out nextDecompressionHandlerLet's first look at the most core source codeSendAsyncMethod, this method is the execution method of sending a request
internal override async ValueTask<HttpResponseMessage> SendAsync(HttpRequestMessage request, bool async, CancellationToken cancellationToken)
{
//Judge whether it is a GZIP compression request. If so, add the request header Accept-Encoding header to gzip if (GZipEnabled && !(s_gzipHeaderValue))
{
(s_gzipHeaderValue);
}
//Judge whether it is a Deflate compression request. If so, add the request header Accept-Encoding header to deflate if (DeflateEnabled && !(s_deflateHeaderValue))
{
(s_deflateHeaderValue);
}
//Judge whether it is a Brotli compression request. If so, add the request header Accept-Encoding header to brotli if (BrotliEnabled && !(s_brotliHeaderValue))
{
(s_brotliHeaderValue);
}
//Send a request HttpResponseMessage response = await _innerHandler.SendAsync(request, async, cancellationToken).ConfigureAwait(false);
( != null);
//Get the returned Content-Encoding output header information ICollection<string> contentEncodings = ;
if ( > 0)
{
string? last = null;
//Get the last value foreach (string encoding in contentEncodings)
{
last = encoding;
}
//Judge whether the server uses gzip compression based on the response header if (GZipEnabled && last == Gzip)
{
//Use gzip decompression algorithm to decompress the return content and assign it from the new value to = new GZipDecompressedContent();
}
//Judge whether the server uses deflate compression based on the response header else if (DeflateEnabled && last == Deflate)
{
//Use deflate decompression algorithm to decompress the return content and assign it from the new value to = new DeflateDecompressedContent();
}
//Judge whether the server uses brotli compression based on the response header else if (BrotliEnabled && last == Brotli)
{
//Use brotli decompression algorithm to decompress the return content and assign it from the new value to = new BrotliDecompressedContent();
}
}
return response;
}
Through the above logic we can seeGZipEnabled、DeflateEnabled、BrotliEnabledThe three variables of bool type determine which request compression method is adopted. The main implementation method is
internal bool GZipEnabled => (_decompressionMethods & ) != 0; internal bool DeflateEnabled => (_decompressionMethods & ) != 0; internal bool BrotliEnabled => (_decompressionMethods & ) != 0;
It is mainly based on our configurationDecompressionMethodsEnumeration value determines which method of compression result to obtain, and the implementation logic of decompression is encapsulated inGZipDecompressedContent、DeflateDecompressedContent、BrotliDecompressedContentmiddle,Let's take a look at their specific code
private sealed class GZipDecompressedContent : DecompressedContent
{
public GZipDecompressedContent(HttpContent originalContent)
: base(originalContent)
{ }
//Use the GZipStream class to decompress the returned stream protected override Stream GetDecompressedStream(Stream originalStream) =>
new GZipStream(originalStream, );
}
private sealed class DeflateDecompressedContent : DecompressedContent
{
public DeflateDecompressedContent(HttpContent originalContent)
: base(originalContent)
{ }
//Use the DeflateStream class to decompress the returned stream protected override Stream GetDecompressedStream(Stream originalStream) =>
new DeflateStream(originalStream, );
}
private sealed class BrotliDecompressedContent : DecompressedContent
{
public BrotliDecompressedContent(HttpContent originalContent) :
base(originalContent)
{ }
//Use BrotliStream class to decompress the returned stream protected override Stream GetDecompressedStream(Stream originalStream) =>
new BrotliStream(originalStream, );
}
}
The main working method is to use the decompression method of the corresponding compression algorithm to obtain the original information. To summarize briefly,HttpClientThe compression-related processing mechanism is, first according to your configurationDecompressionMethodsDetermine which compression algorithm you want to use. Then match the corresponding compression algorithm and add Accept-EncodingThe request header is the compression algorithm you expect. Finally, obtain based on the response resultContent-EncodingOutput header information, determine which compression algorithm is used by the server, and decompress the original data using the corresponding decompression method.
Summarize:
Through this discussionHttpClientWe can learn about the processing of response compression,HttpClientThere are very high flexibility and scalability in both design and implementation methods, which is why.Net CoreOfficially recommended onlyUse HttpClientA way to request Http. Since it is simple to use and clearer to implement it, I will not explain it too much here. Mainly I want to tell everyoneHttpClientBy default, the response compression can be processed directly, rather than as we used beforeHttpWebRequestIt also needs to be implemented manually.