SEO & Ajax
How do you provide your customers with the Ajax functionality they want and still optimize the site for search engines?
The Problem
You customer has a site that ranks very high in Google but it's pretty static with not much Flash and even less in the way of web 2.0 functionality (Ajax), meaning none. They are also B2C and their target audience is mostly men between the ages of 18-40 and gadget freaks. This mean at the minimum the sight needs to look cool, but more and they also want it to function cool (Ajax) without losing any of the hard earned page rank. The customer also sees all the Ajax stuff and likes it, and lets face it, it does make sense when looking at products to only load the different information (specs, related products, etc) without a page refresh. So how do you bring the power of Ajax into play without cutting of the crawlers ability to follow and index?Possible Solution
I've done a bit of research on this topic because of the above problem and have come to the following realization.100% Ajax and SEO don't mix
You will notice I said 100% Ajax. That means your site still needs to produce some content even with Javascript completely disabled even if it looks like crap, or made invisible with css. The main thing is that a crawler needs to be able read/index and follow links.
Take for instance your classic tab panel produced by ExtJS the popular Ajax framework. If you disable Javascript and refresh the page you end up with nothing. But if you view the source you will notice that there is indeed some content on the page. The first tab panel was built with existing content. Now this doesn't help you much since you want to load the data later using Ajax when a user clicks on a tab. But generally you set the first tab to be the default tab which would then immediately grab the data, so why not output the data directly in the first tab.
Here is the markup for the first tab.
<script>
Ext.onReady(function(){
// basic tabs 1, built from existing content
var tabs = new Ext.TabPanel({
renderTo: 'tabs1',
width:450,
activeTab: 0,
frame:true,
defaults:{autoHeight: true},
items:[
{contentEl:'script', title: 'Short Text'},
{contentEl:'markup', title: 'Long Text'}
]
});
});
</script>
<div id="tabs1">
<div id="script" class="x-hide-display">
<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed metus nibh, sodales a, porta at, vulputate eget, dui. Pellentesque ut nisl. Maecenas tortor turpis, interdum non, sodales non, iaculis ac, lacus.<br/><br/> Vestibulum auctor, tortor quis iaculis malesuada, libero lectus bibendum purus, sit amet tincidunt quam turpis vel lacus. In pellentesque nisl non sem. Suspendisse nunc sem, pretium eget, cursus a, fringilla vel, urna.</p>
</div>
<div id="markup" class="x-hide-display">
<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed metus nibh, sodales a, porta at, vulputate eget, dui. Pellentesque ut nisl. Maecenas tortor turpis, interdum non, sodales non, iaculis ac, lacus. Vestibulum auctor, tortor quis iaculis malesuada, libero lectus bibendum purus, sit amet tincidunt quam turpis vel lacus. In pellentesque nisl non sem. Suspendisse nunc sem, pretium eget, cursus a, fringilla vel, urna.<br/><br/>Aliquam commodo ullamcorper erat. Nullam vel justo in neque porttitor laoreet. Aenean lacus dui, consequat eu, adipiscing eget, nonummy non, nisi. Morbi nunc est, dignissim non, ornare sed, luctus eu, massa. Vivamus eget quam. Vivamus tincidunt diam nec urna. Curabitur velit.</p>
</div>
</div>
This has all the data loaded on page and uses the framework to "pretty" it up. But we could also do something like this.
<script>
Ext.onReady(function(){
// basic tabs 1, built from existing content
var tabs = new Ext.TabPanel({
renderTo: 'tabs1',
width:450,
activeTab: 0,
frame:true,
defaults:{autoHeight: true},
items:[
{contentEl:'script', title: 'Short Text'},
{contentEl:'markup', title: 'Long Text',autoLoad:{url:'content.cfm?page=2')
}
]
});
});
</script>
<div id="tabs1">
<div id="script" class="x-hide-display">
<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed metus nibh, sodales a, porta at, vulputate eget, dui. Pellentesque ut nisl. Maecenas tortor turpis, interdum non, sodales non, iaculis ac, lacus.<br/><br/> Vestibulum auctor, tortor quis iaculis malesuada, libero lectus bibendum purus, sit amet tincidunt quam turpis vel lacus. In pellentesque nisl non sem. Suspendisse nunc sem, pretium eget, cursus a, fringilla vel, urna.</p>
</div>
<div id="markup" class="x-hide-display">
<a href="content.cfm?page=2" class="x-hide-display">Long Text</a>
</div>
</div>
In the above code I added the autoLoad to the script for the second tab. In the second tab I also removed all the content and placed a hidden URL which a crawler could then follow to obtain the snippet. A user with Javascript enabled would of course notice nothing different, and we have suddenly provided a means for a crawler to index content that would normally not be accessible. To test I used Xenu link sleuth as my crawler and it was able to follow the link without issue.
Conclusion
This isn't a new technique, I found someone alluding to this in a forum somewhere but thought that it deserved some attention and also wanted to find out what you have done to keep your sites SEO optimized while adding in a sprinkling of Ajax goodness.
Subscribe
Subscribe via RSS
Follow me on Twitter
Or, Receive daily updates via email.
Tags
adobe air ajax apple cf community cfml coldfusion examples ext flash flex google javascript max2007 max2008 misc open source programming railo software technology ui
Recent Entries
Converting structkeys to lowercase
Blogroll
An Architect's View
CFSilence
Rey Bango
TalkingTree
Wish List
Tabsets are similar... they're more flexible and powerful than something like the ext tabset. They look like a tabset when they load with or without JS and they don't foul up screen-readers either.
Do you have a few working examples that you could link?
@Glyn,
The above code example does just that, I checked it with a page crawler and it followed the invisible links, and thats just the point, you need to be able to provide the spiders the content without relying on JavaScript, since they ignore it.
with all the Ajax'ified websites out there that just don't provide a noscript version of the site they completely lose out, their site doesn't get indexed, and if it doesn't get indexed it doesn't show up in searches, and if your business is made either by PE's or by high google rank then you will be out of business in no time.
Accessibility is important but high search engine ranking is everything for a lot of Internet based businesses.
however if the content is rendered (like in this example) on the page then reguardless of what you do with the data/text Google will always see it, you can hide it behind AJAX stuff and as long as the text is present it will be seen.
the markup in Gary Gilbert example for the first tab WILL BE SEEN BY GOOGLE..... HOWEVER Google may think he is hiding text because of the CSS 'display hide' style and would not take kindly to this approach and may not index it at all. often people make the mistake of thinking that the reason the page was not index was because of the javascript but in fact it had nothing to do with it.
if you turn off Javascript thats how Google will see your site, but Google's advanced algorithms look for CSS and styles to see if the user is spaming or hiding something using styles the first example could get trapped by this rule. the second example is no better. (this is all depended on the CSS being the display hide)
all that being said it would be irresponsible for me not to common on AJAX and accessibility which it a different story and had other factors on rank
You raise a good point. In order to prevent out of context parsing the snippet of code in the non-javascript version must contain the same meta data as the main page, since in actuality the content snippet is in fact a sub-part of the main page. This would therefore should prevent any out of context parsing or?
You were right but this is no longer true, now with ItsNat you can build Single Page Interface web sites AJAX intensive and in the same time with no duplication, the same site page based when JavaScript is disabled or ignored like in web crawlers.
Take a look:
http://itsnat.sourceforge.net/php/spim/spi_manifes...
http://itsnat.sourceforge.net/index.php?_page=supp...
http://www.innowhere.com:8080/spitut/ (online demo)