From the Dogpiles to the AltaVistas, search engines have come a long way indeed. Some are now defunct, some are struggling while others continue thriving. The current big guys are Google, Bing and Yahoo. We humans, since time immemorial, have always had this strong inner desire to know. This curiosity and courage to go beyond our limits. It’s true we have plenty of problems here on earth, but that won’t stop us from exploring the moon or outer space. We scan the heavens for celestial bodies that are millions of light years away. We try to determine the edge of the Universe. We are obsessed with trying to understand black holes. Or quantum physics. Why? Why all these? Well, because we want answers. It’s why we try every day to search for answers. We are hard-wired to search for answers. Where did we come from? Why are we here? How did the Universe begin? When and how will it end?
We humans are curious by nature. Some people think that’s why our brains have developed so well compared to other creatures. We push ourselves to the limit. It’s why we search. Not only do we search for heavenly answers, we also search for answers right within our backyard here on earth? Answers to common day to day questions. Like who scored for Manchester United versus Ajax? Or how can we format a HTML document? Or who created WordPress? It’s why we have search engines, to help us answer questions like these.
The most popular search engine is Google. It was created by Larry Page and Sergey Brin. Initially it was called Backrub. From the moment of its creation, it stood out from the rest. It was different. While other search engines focused on the obvious keyword relevance, Backrub used the number of backlinks to a page to determine the search engine ranking. It’s now obviously much more complex than that. However, it has demonstrated how important it is to have a good search engine. A search engine that gives you intelligent answers. Google is now one of the most valuable companies in the world. It is the search engine that’s contributed to that more than any of their wide range of products.
We are not going to develop another search engine to compete with Google today. However, before we think that big, we need to look inwards first: our website’s search engine.
WordPress has an inbuilt search engine and it’s nice, it works. However, for a large site, you may want to be more ambitious. In relation to the search functionality we can get from some free plugins, wordpress’ default search is just too basic. The only way to customize it is to get your hands greasy with code.
# Objective of this Lesson
The main of objective of this lesson is to:
- Build a powerful search engine that’s not only accurate but also gives us control of the logic used to generate the results. This means we can experiment with several customizations to see which one is accurate for our users.
We assume that you:
- Already have a working wordpress installation, hosted either remotely or locally. You may also need internet connectivity to install our plugin, which is actually a few Kilobytes.
Why we are using Relevansi?
The plugin we are using is called Relevansi. It’s the most popular search plugin as of now. I'm not surprised by that as I have used it in my personal projects. Here’s why we are using it:
- It’s free and easy to use.
- You build the search index manually. This then gives us control over the content we would like to be searchable. We can for instance leave some custom post types or taxonomies out of our index.
- It can expand shortcodes. I love this. I really do. It’s probably the number one reason I discovered relevansi last year. What we mean here is that when searching, Relevansi will execute the content of shortcode and index the result as well. For instance, say you are fetching some data from a third party site via an API, and you need this data searchable to users. In the default search or many other plugins, that data won’t be available. It’s not in your database, so users won’t be able to get any results. However, with Relevansi, that data will be factored in the search results as well. Not only that but you can also disable this feature as we shall see later.
- It automatically scans and lists stopwords for you. Moreover, you can also add the stopwords yourself. But what are stopwords? Mmm! They are useless words. These are words that appear in any every document and don’t really influence search results. Examples: a, the, me, again, all, can etc. Furthermore, removing them makes the index much smaller hence improving search performance.
- It also provides something seemingly simple but extremely powerful: default operator for search, that is ‘OR’ and ‘AND’ for your queries. ‘OR’ is used when any term is present while in ‘AND’ all terms are present. A simple switch of either of these two causes massive differences in your results. Try it out.
- And hey, its default order for results is another reason we are using it. Relevansi defaults to relevance first, not date, however you can switch to what you like.
- You can also limit search results. If you do so relevansi limits to the first 500 results.
- And one of the best features right here: weights. Relevansi allows you to assign weights to various post types and taxonomies. You do this simply by entering integers. Say you want to give more priority to some custom post types, then assign more weight to that custom post type. This is great, I have seen some search plugins which don’t even include custom post types in their results, let alone assigning weights.
- You can choose whether to index post authors, post excerpts and comments as well.
- Look, the advantages are many. Let’s proceed to installation.
To install relevansi:
- Head over to the plugins directory. Type ‘relevansi’. Search.
- Install it by clicking ‘Install Now’ button.
- Activate it.
The best thing about Relevansi is that it has plenty of really useful and powerful features that can enhance our search functionality. We have listed majority of these features in the ‘Advantages’ section above. Now settings page is just a matter of enabling or disabling a given option.
- First let’s build our index. Relevansi search doesn’t work without you building the index. It needs to index your content according to your settings. Just click the ‘Build Index’ button as shown below we proceed.
- Well, my content here has just been indexed.
- Here’s our basic options page. You can change the default operator, limit search results, define search results order etc.
- We also have log settings. We have exclusions as well. We can exclude certain post types, pages, categories or tags.
- We can highlight search results. We can define search highlight CSS.
- We also have indexing options. Choose the post types or taxonomies you want indexed.
We can also choose to include whether to expand shortcode contents during indexing. Moreover we can choose whether to index comments, authors and post excerpts.
Note that:Relevancy will use the default search form as long as you have activated it and built the index.
Let’s see a simple example with highlighting below:
Just like airports are the gateway to any country or city, search engines are gateway to the web. One of the reasons Google has been able to dominate the search market is because of its clean and concise interface. The age of people browsing through multiple pages just looking for some piece of content is over. The answer should always be just a search away.
It’s therefore important you get your personal website’s search engine accurate and relevant. It’s very simple, if it’s not accurate then definitely the user won’t find your content. He’ll go elsewhere of course. A good search engine should be customizable. Different inputs should lead to different results.
We’ve looked at some of the features that make relevansi powerful. Features like assigning weights, manual indexing, stopwords, shortcode expansion etc. You should always build your index as you add more and more data. You don’t have to start from the beginning, you can continue the build from where you left.
The reality is we humans will always continue asking questions, searching for answers. Be it when we look to the heavens, or even when I visit your site. I won’t be browsing through hundreds of pages, I promise. I will search. If you have a good search system, I will find what I want and I will stay and be coming back. If you don’t have it I will go. And God knows what may bring me back. Let’s therefore use plugins like relevansi for better search.