LucidWorks Search on Windows Azure delivers a high-performance search service based on Apache Lucene/Solr open source indexing and search technology. This service enables quick and easy provisioning of Lucene/Solr search functionality on Windows Azure, and today I’m happy to announce that LucidWorks has rolled out support for indexing of Azure Tables and Azure Blobs, while also making the signup process easier than ever, thanks to tight integration with the Windows Azure store.

In this post, we’ll take a look at how easy it is to get started with a Search Application that accesses your indexes on Azure to provide custom search functionality. For more information about working with LucidWorks on Azure, see also the blog posts from Microsoft Open Technologies, Inc, covering some of the key features of LucidWorks, as well as an overview of how to get started.

Signing up for LucidWorks Search via the Windows Azure Store

This is a new feature that we’re excited about.  Along with several other key partners, we’ve enabled LucidWorks customers to set up an instance on Windows Azure directly from the dashboard.  Here’s how easy it is to get started:

First, from the Windows Azure Dashboard, click New, then Store:

Next, you’ll be prompted to choose an Add-on from a list.  Select LucidWorks Search.  The next screen invites you Personalize your new Add-On:

At this point, all you have to do is enter a new Name for your LucidWorks Search Add-on and the region you want your instance to be located in.

Right now there’s only one LucidWorks Search account option, the Micro level, which is great for getting started.  Should you exceed the limits of the Micro level, you can also sign up for other enterprise-level accounts from the LucidWorks Dashboard. 

Once you move to the next screen, you’ll be asked to confirm your selections.  Once selections are confirmed, Windows Azure creates an account for you on LucidWorks’ Web site, where you will manage your LucidWorks Search configuration via a Dashboard.  The process also automatically provisions network and storage for your LucidWorks Search indexes on Windows Azure.

Next, you can log in to your Dashboard to manage your LucidWorks Search configuration by selecting your LucidWorks instance in the list of Add-Ons in the Windows Azure Dashboard menu, then clicking on the link under “Manage your Add-On”:  

As I mentioned in the Introduction, there are previous blog posts to refer to for info on LucidWorks’ features and how to get started with the LucidWorks Dashboard, but today I want to make sure you know about two new data sources that are Available in LucidWorks Search on Windows Azure.

LucidWorks support for Windows Azure Tables and Windows Azure Blobs

Along with the Windows Azure store integration, we also released LucidWorks Search support for Windows Azure Blobs at Table Storage in our latest release.  Both are available via the LucidWorks Search Dashboard under Indexing > Data Sources:

Windows Azure Blobs provide a way to store large amounts of unstructured, binary data, such as video, audio, and images, including streaming content such as video or audio. There are two types of blob storage available, block blobs and page blobs. Block blobs are optimized for streaming and referenced by a unique block ID.  Page blobs are optimized for random access and composed of pages that are referenced by offsets from the beginning of the blob.  More information on Windows Azure blobs can be found here.

Windows Azure Table storage is a collection of non-relational structured data.  Unlike tables in a database, there is no schema that enforces a certain set of values on all the rows within a table. Windows Azure Storage tables are more like rows within a spreadsheet application such as Excel than rows within a database such as SQL Server. Each row can contain a different number of columns, and of different data types, than the other rows in the same table.  You can find more information on Windows Azure Table storage here.

Sample application: Accessing LucidWorks Search Indexes on Windows Azure

Next I want to show you a working example of an ASP.NET application customized for a LucidWorks Search solution.  LucidWorks has several examples of client applications to get you started, including documentation and sample code for connecting to LucidWorks Search indexes from Perl, Python and .NET. 

This example shows you how easy it is to set up an application in Visual Studio to connect to LucidWorks indexes using ASP.NET and C#.  It’s currently running on a Windows Azure Web Site that accesses a LucidWorks Search Index that is also running on Windows Azure.

Here’s a layout of the architecture:

In this example, application code on a Windows Azure Web site makes calls to LucidWorks on Windows Azure, which accesses the Index in storage and returns results.

For a sample index, we created Web Site data sources for the windowsazure.com Web site and the lucidworks.com Web site. Both were combined and searchable in our LucidWorks Search on Windows Azure instance.  Here’s the lucidworks.com Web Site data source configuration:

To get started, we created a Visual C# ASP.NET Web forms application project in Visual Studio by selecting File > New Project and choosing the Visual C# Template option:

This creates a new project that can be run right away locally, and has plenty of great features like mobile interfaces and automatic component resizing.  Here’s what it looks like with no code changes:

And here’s what the customized Web application looks like:

The ASP.NET Web forms application is a great way to get started in building a LucidWorks Search solution that runs on Windows Azure Web Sites.  Making the ASP.NET Web forms application into a LucidWorks Search application required just a few changes to a few of the provided template components. 

First, Default.aspx was edited to serve two content sections; one for the search request and another for the search results in a List View. 

The code that actually performs the search and returns the results is in Default.aspx.cs.  First, when the user hits enter, the RunSearch class is called. 

RunSearch first gets the last search word from the session, if there is one:

   privatevoid RunSearch(string SearchWord)

        {

            List<SearchResultsItem> FoundItems = newList<SearchResultsItem>();

            try

            {

                Session["SearchWord"] = SearchWord;

                hdrResultsHeader.Visible = false;

Next, the Network Credentials that are used to authenticate with the LucidWorks Search Service are retrieved from the project’s Resources.resx file.  LucidWorks provides each Search Instance with an API User ID, Password and unique URL for searching indexes, which can be obtained via the LucidWorks Dashboard.  In this case we pass these values along with the search keyword to the LucidWorks Search Instance using REST:

NetworkCredential MyNetworkCredentials = newNetworkCredential(Resources.User, Resources.Password);

HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create(Resources.LucidWorksLink + SearchWord);

We then establish that this is a HTTP GET request that expects results back in XML format. You can also pass values back in JSON, but parsing and formatting is easier to perform on the XML results. 

               WebReq.Credentials = MyNetworkCredentials;

               WebReq.Method = "GET";

               WebReq.ContentType = "application/xml";

A response object and an XML document object are instantiated.  Each search result node is defined using XPATH at /response/result/doc in the search results.

               HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();

               XmlDocument xmlDoc = newXmlDocument();

               xmlDoc.Load(WebResp.GetResponseStream());

               XmlNodeList ItemNodes = xmlDoc.SelectNodes("/response/result/doc");

Next, for each node, we parse the results and place them in the calling page’s list view by parsing a node’s attributes to determine what part of the results is returned.  Nodes with the attribute “title” are the result item title, IDs are (in this case) URLS, and descriptions are descriptions of the search result item.  Once found items are extracted they are added to the results list:

               foreach (XmlNode Item in ItemNodes)

               {

                 SearchResultsItem FoundItem = newSearchResultsItem();

FoundItem.Title = ExtractString("<arr name=\"title\"><str>", "</str>", Item);

FoundItem.Description = ExtractString("<str name=\"description\">", "</str>", Item);

FoundItem.Link = ExtractString("<str name=\"id\">", "</str>", Item);

                    FoundItems.Add(FoundItem);

                }

            }

For errors in string handling, bad requests, down servers or anything else, an empty list is returned to the results Container and the user will see a no data found message. Otherwise the search result data is bound to the Web page and results are displayed:

            catch (System.Exception)

            {

            }

            lvSearchResults.DataSource = FoundItems;

            lvSearchResults.DataBind();

        }

We also made some cosmetic changes to the design of the site, like removing some tabs at the top of the page that didn’t apply for our solution, but these were mostly cosmetic alterations that you can see here in the code sample on GitHub.  Here’s what the results look like in the search window:

Because we used the ASP.NET Web forms application as a template for this solution, there is also built-in functionality to handle security and a few other features (there are a variety of security options at the app level and in the LucidWorks instance as well), but this is a good start as an example of what you can do with minimal effort.

Publishing your Web Application Code to a Windows Azure Web Site

Once you have the Web application working the way you’d like it to work, the next step is to publish it to a Windows Azure Web site.  This is easily facilitated by the Windows Azure publishing feature in Visual Studio, 

First step is to set up a new Windows Azure Web site by going to your Windows Azure Dashboard and creating an empty Web site by selecting New > Compute > Web Site > Basic Web Site.  Once the new site is created, under “Quick Glance” on the right, select “Download Publish Profile”.

Next, in Visual Studio, right click on the solution file object and select Publish…:

Next, simply import the publish profile you just downloaded from your empty site and press the Publish button.  That’s all it takes to get your code up to the Web!

Once the publish profile has been imported the first time, it’s stored with the project.  The next time publishing is a matter of right clicking on the solution file object and selecting Publish…, then confirming your intent to publish to your Windows Azure Web site. .

Summary

I’ve shared some simple examples of the sort of thing that you can do with LucidWorks Search on Windows Azure, and how you can easily and quickly set up an application in Visual Studio to build full-featured search applications.  Note that you could add data sources and configurations via the LucidWorks Dashboard that would automatically become part of the search application when indexes are updated, with no changes to the Web Application’s source code.    Get started with your own multi-tier LucidWorks Search solution by signing up for a LucidWorks Add-on, configuring your LucidWorks search sources,  and downloading and publishing the sample code here, and let us know what you think!

The post Tutorial: Setting up a LucidWorks Search Service on Windows Azure appeared first on MS OpenTech.