HTML page retrieval from a site requireing
Authentication

One of the major pieces of functionality i almost allways need is the ability to retrieve a file from a website where the website needs credentials for the purpose of authentication.

As part of the System.Net.WebRequest object configure the WebRequest.Credentials and WebRequest.Certificates properties with the necessary authentication information.

Based on the websites authentication approach we need to do different things to set these properties.

1) Basic/Digest, transmit a username and password by manually creating System.Net.NetworkCredential object and assigning it to the WebRequest.Credentials. Digest you may also supply the domain name.

2) Windows Integrated same approach as above or use the current logged in user from System.Net.CredentialCache.DefaultCredentials

3) Client Certificates rquired, load from a file using System.Security.Cryptography.X509Certificates.X509Certificate2 and add it to the HTTPWebRequest.ClientCertificates collection.

4) You can load an X.509 certificate from a certificate store using the System.Security.Cryptography.X509Certificates.X509Store found in the System.Security.dll. [this was introduced in .NET 2.0]. You can search the store X509Store.Certificates.Find or present the user with a list to choose from via the SelectFromCollection method on System.Security.Cryptography.X509Certificates.X509Certificate2UI.


Sample code


using System.Net;
using System.Security.Cryptography.X509Certificates;

//BASIC Authentication
WebRequest requestA = WebRequest.Create("http://www.RipThatPage.com");
requestA.Credentials = new NetworkCredential("username", "password");
requestB.PreAuthenticate = true;

//WINDOWS Authentication
WebRequest requestB = WebRequest.Create("http://www.RipThatPage.com);
requestB.Credentials = CredentialCache.DefaultCredentials;
requestB.PreAuthenticate = true;

//CLIENT CERTIFICATE
HttpWebRequest requestC = (HttpWebRequest)WebRequest.Create("http://www.RipThatPage.com);
X509Certificate cert1 = X509Certificate.CreateFromCertFile(@"..\..\TestCert.cer");
requestC.ClientCertificates.Add(cert1);