The Trinity of Technical SEO: Latency, Indexation and Bandwidth
As most of us know, SEO goes way beyond Titles, Tags and keyword selection. Gone are the days where heavily optimizing a page for “crazy purple widgets” will get you where you want to be. As the Search Engines focus on their core goal of providing the most authoritative and relevant results for each query we as Internet Marketers are faced with a new challenge. As web technology progresses, we must look beyond what a site “says” and look at what a site “is”. Structure and construction become even more important.

Site Latency: How fast do your site pages load?
Let’s first start with Latency. The Latency of a site is the speed in which the page loads for a visitor. There are many factors that contribute to the Latency of a web page including, but not limited to, site construction, reliance on external resources, server capacity, page size, etc. We do our best to reach a happy medium with the factors we can control. Not every site can afford Google-size processors and server capacity; most are sharing their server with hundreds or thousands of other sites. When this is the case we have to focus on factors we have more of an influence on like how our site is constructed and how bulky it is. In the end there is only so much speed you can get out of your site for your given budget.

How does one assess site Latency?
There are many tools out there that can be used for testing the latency of a site. Some are paid which we use extensively, and some are free, which we also use extensively. One of the most useful resources is Google Webmaster Tools. In April 2010, Google officially announced they would be incorporating site speed as one of the 200+ site signals used in determining search rankings and now have become sort of the de facto authority on this topic. Let’s use an actual site example where we are using Google Webmaster tools.

This is a site that relies very heavily on external resources, with a lot of multimedia. There is no way around this, so the speed of the site is limited by the speed of anywhere from 6-12 other servers to provide it with the data it needs to load a page. Here is a screenshot from Google Webmaster tools showing the time it takes to load a page:

As you can see, over the last 90 days the sites pages load on average just over 4 seconds. This is a site that has been receiving exponentially more traffic over this time frame. I want to also make the point that in the third week of June the site was redesigned and recoded for a better user experience and greater efficiency. As such, you can see the line is very erratic until July when the spikes smooth out a bit and on average the time has decreased. Remember we are only talking a spread of 1.3 seconds between the high and low. Not a huge amount off is it? Most visitors wouldn’t even notice a difference. But Search Engines do.

This brings us to Bandwidth.
Bandwidth is defined as “the maximum amount of information (bits/second) that can be transmitted along a channel”. Why does this matter? Search Engines have a limited amount of capacity available. Granted, their “limited capacity” far exceeds anything most of us can dream of, but it is also a really big Internet to crawl (over a trillion pages). Based on this, the Search Engines will allocate a certain amount of resources to crawling a particular website based on its perceived value (whether it is stated or not). CNN.com is going to get a substantially larger portion of Google’s resources than my Dog in Funny Hats blog.

So what does this mean to the rest of us? It means that we have to make the best use of the resources that are given to us. Basically, when Google comes a-knockin’ it is in your best interest to make sure it has the most clear path through your site and can get as much information as possible before it leaves for its next appointment (probably cnn.com). This is why building the most search-friendly efficient site is critical.

Below is a screenshot of the same site, same time period as before, this time showing how much bandwidth is afforded this site by Google in a given day:

Notice the trend of increasing bandwidth up until the third week of June, when the new site design was launched. Immediately before the 2.0 version of the site was launched the site was receiving a peak high of almost 89,000KB of attention from Google. Then it decreased substantially and immediately to 12,000KB and has since settled in somewhere around 39,000KB. The initial impulse is to look at this and say that a mistake was made, Google isn’t as interested in crawling this site anymore. This next screenshot shows the truth – how many pages is Google crawling in a given day:

This chart pretty much speaks for itself. Based on this chart we can see, that while the Latency of the site has only moved within a narrow band, the bandwidth usage has dropped considerably, which has allowed Google to crawl more pages on a given visit. This is a strong case for optimizing your entire site presence, not just your Titles and Tags. During this time period of this example, no on-site SEO elements were changed.

And by the way this third chart mimics the organic site traffic trend as well. How well is your site performing, contact us today to conduct a Technical Site Assessment and start improving or rebuilding your online presence?

A Short How-To On Identifying Indexation Problems during an SEO Audit
If you think you might be having trouble with indexation there are some simple checks to do.

1. Do a physical check of the pages and site structure. This could be done with a database tool or by hand, thus giving you a chance to review the copy of your site, because most likely you haven’t done that in a while.

2. Run a XML sitemap of domain.com to get a list of the URLs present on the site. It is important to use a web-based or desktop application to do this and not rely on an internal tool that generates the sitemap from a CMS database. Why you ask? External sitemap generators (Web-based or desktop) do not have access to the server so they must crawl the site from link to link, just like a search engine. This will give you a better understanding of what content is accessible and what is not.

3. Check your Analytics program. Run a report of all content on the site that has received non-paid Search Engine traffic over a period of time (how long depends on your site traffic levels).

4. Query “site:domain.com” in any Search Engine to get a list of the URLs from that domain that are included in the index. Also, check the www version of the site to see if there is any variance. Theoretically, these numbers should be near identical.

Now that you have this data, what do you do with it?
Now that you have this data, compare them to each other. In Step 1 you identified all the pages/URLs that exist on the site. Now, compare this list to Step 2 – if not all pages that physically exist on the site are present in the sitemap, then you have some investigation to do. This indicates that there may be some issue with the structure of the site that is preventing crawlers from reaching those pages.

Next, compare Step 3 with Step 1.
Are there pages that are present on the page but have never received any traffic from Search Engines? This is an indication that these pages may not be indexed by the Search Engines.

There are pages that you don’t really want traffic to, like your refund policy, or your list of pending lawsuits (note: if you have this on your site take it off). If your product or services page is not receiving traffic this is something that should be addressed.

Finally, compare Step 4 with Step 1.
What pages on the site are not indexed? If you see that a large number of pages are not included in Step 4, you may have an indexation issue. As a method for double-checking this issue, compare the pages missing from Step 4 with Step 3. If there are pages that are not currently in the index, but have gotten Search traffic in the past, these pages may have gotten de-indexed for some reason. Investigate why this might be, especially if you consider them mission-critical pages. Don’t overlook checking your robots.txt file. It is not uncommon to see large sections of a site disallowed, when they shouldn’t be.

How well is your site performing, contact us today to conduct a Technical SEO Site Assessment and start improving or rebuilding your online presence for maximum visibility?

Whether novice or pro, the annotations feature in Google Analytics (GA) is one feature your entire organization should be using. Earlier this year it became available to all GA account users and is one of the most simple features to use. Not only can it have a huge impact on tracking and monitoring site traffic, it will save you eons of time when going back historically to understand what components had the highest influence on generating traffic to and actions on your site.

What is the Annotation Feature in Google Analytics?
The annotation feature allows GA users to add notes to a website profiles’ traffic, they can share these with other users of this profile or mark them private. See example below. Notations were made to indicate when internal IP traffic was filtered, when an action to promote the site was performed and when campaigns launched.

What are the benefits of this feature and how can they make a difference for you and your company?
There are many spokes to an organization’s wheel such as PR, marketing, and customer service. In addition, there are the spokes that are not entirely under our control like social media channels. Since there are so many moving parts in an organization it is difficult to keep track of what may be influencing or impacting your daily web traffic.

The annotation feature in GA allows users to tell a story through site statistics. Perhaps major site changes were made, a new campaign was launched or a high profile blogger mentioned your company. These are just a few of the types of items that can be notated in the time-length of your web stats in GA. The possibilities are endless when logging and reviewing quarterly activity within your company – what happened this month, last month, the past 90 days? By allowing all departments to add notes to the site traffic they can provide insight into how their efforts are effecting the overall mission and understand which activities were most successful.

Think of annotations as a shortcut to digging into that spike in traffic or boost in sales. If you can follow traffic on a specific date or length of time when a specific campaign was launched you will be providing yourself and your team with a visual on its performance.

I’ve included a 58 second demo video from Google on their annotation feature below, enjoy and get to posting the notations in your stats regularly for the benefit of your entire team.