Better way of handling sitecore 404 pages

Out of the box Sitecore way of handling 404 error is to respond with 302 redirect the original request to "sitecore/service/notfound.aspx". Then "sitecore/service/notfound.aspx" will respond with 404 response code.

It is not a good way for SEO purposes as crawlers think the page exists. It did not know it is a 404 response.

Sitecore provides a configuration setting not to redirect on an item not found.

 <setting name="RequestErrors.UseServerSideRedirect">
        <patch:attribute name="value">true</patch:attribute>
 </setting>

 Changing RequestErrors.UseServerSideRedirect to true will not redirect but with the response with 404 status code.

This helps in some way, but not completely, as in Sitecore we want 404 page to be content manage.

 So, on the page not found 

• we need to respond with 404.
• Do a server tranfer to sitecore 404 content page.
• reposnds 200 status code of 404 content page.

There is a configuration setting in Sitecore to set 404 content pages in Sitecore:

 <setting name="ItemNotFoundUrl">
        <patch:attribute name="value">/404</patch:attribute>
      </setting>
      <setting name="LinkItemNotFoundUrl">
        <patch:attribute name="value">/404</patch:attribute>
 </setting>

We can set the name of 404 content page here.

Here is how we can achieve this:

• Add a processor after HttpRequest.ItemResolver on HttpRequestBegin pipeline.
• Find if the content item is null if null get 404 content page set as the context item.
• Add another processor in the pipeline httpRequestEnd after EndDiagnostics to set the 404 status code for not found Item.

Adding a processor after HttpRequest.ItemResolver on HttpRequestBegin pipeline

Add code to your patch file

<pipelines>
<httpRequestBegin>
        <processor type="sitecoreBlog.tools4geeks.SitecoreCustom.Pipelines.HttpRequest.NotFoundProcessor, sitecoreBlog.tools4geeks.SitecoreCustom"
             patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']" />
      </httpRequestBegin>
</pipelines>

public class NotFoundProcessor : Sitecore.Pipelines.HttpRequest.HttpRequestProcessor
    {
        public override void Process(Sitecore.Pipelines.HttpRequest.HttpRequestArgs args)
        {
            if (Sitecore.Context.Item != null || Sitecore.Context.Site == null || Sitecore.Context.Database == null)
            {
                return;
            }

            if (Sitecore.Context.Item == null)
            {
                var notFoundItem =
                    Sitecore.Context.Database.GetItem(string.Concat(Sitecore.Context.Site.StartPath,
                        Settings.ItemNotFoundUrl));
                if (notFoundItem != null)
                {
                    CustomContext.Current.PageNotFound = true;
                    Sitecore.Context.Item = notFoundItem;
                }
            }
        }
	}

Inherit your NotFoundProcessor from HttpRequestProcessor and override the process method.

I have created a singleton class called CustomContext to store values on every HTTP request, where I have stored current context request is PageNotFound.

Adding processor in the pipeline httpRequestEnd after EndDiagnostics

Add this to your patch file:

<pipelines>
	<httpRequestEnd>
        <processor type="sitecoreBlog.tools4geeks.SitecoreCustom.Pipelines.HttpRequest.SetHttpResponseCode, sitecoreBlog.tools4geeks.SitecoreCustom"
             patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.EndDiagnostics, Sitecore.Kernel']"/>
	</httpRequestEnd>
</pipelines>

public class SetHttpResponseCode : HttpRequestProcessor
    {
        public override void Process(HttpRequestArgs args)
        {
            if (CustomContext.Current.PageNotFound)
            {
                HttpContext.Current.Response.StatusCode = (int) HttpStatusCode.NotFound;
            }
        }
    }

Here we are checking the value we set on HttpRequestBegin Pipeline CustomContext.Current.PageNotFound based on it we are setting response status code to 404.

Let me know if you have any questions. Hope this helps.