How To Debug a FAST Search Pipeline Extension

April 28, 2014

Developing a FAST Search Pipeline extension for SharePoint FAST Search is a good way to get FAST to crawl custom properties for files that it would not normally crawl. One issue with it is that trying to debug it isn’t intuitive. I am going to give you some pointers on how you can perform this task.

There are two ways you can do this. The first way is to debug using dummy files that are already set up and you pass as arguments into the program. The second way is to debug when a crawl is running.

Option 1:

  1. Create two files a) output.xml and b) input.xml. Input xml will have the crawled property that you will get your info from. Output will have the crawled properties that you are setting info for.
  2. Open your fast pipeline application.
  3. Edit the properties of your project, and under Debug -> Start Options set the Command line arguments to the location of input.xml and output.xml.
  4. Set a breakpoint and run your application. Now you can debug using your test files.

Option 2:

  1. Open your fast pipeline application, in your Main() in Program.cs place
  1. Build your solution and deploy the new code to your FAST Search pipeline location on your local machine.
  2. Start a full FAST crawl that will use this extension.
  3. Open Task Manager select the processes tab and wait for yourapplication.exe to show up in the list. If it isn’t showing up, make sure you have ‘Show processes from all users’ selected.
  4. When your application shows up, go to your solution, Debug -> Attach to Process.
  5. Find your application in the process list.
  6. Place a breakpoint after the #endif and wait for the breakpoint to hit.
  7. Now you will be able to debug the application during a full crawl.

Being able to debug this in Visual Studio will be very beneficial in determining what is going on with your pipeline. I hope this helps!

Decoding HTML with jQuery

In many of our SharePoint modal dialogs, the back-end needs to send some JSON to the front-end when the dialog is closed.  However, this JSON string gets HTML encoded so something like this:

{"name": "John"}

turns into this:


This makes it difficult for the front-end JavaScript to turn that string back into a JSON object since jQuery.parseJSON() will complain about the string and throw errors. The string needs to be decoded before it can be parsed into a JSON object.


Since JavaScript/jQuery does not have an existing method to do this, we can use this simple trick to decode HTML:

var decoded = jQuery('<div/>').html(value).text();
var json = jQuery.parseJSON(decoded);

A <div> element is created in memory but is never appended to the document. The innerHTML of this element is set to the HTML encoded string, and then its innerText is retrieved. The resulting string is decoded and can then be parsed into a JSON object.

Of course, this trick is useful for decoding any HTML encoded string – not just JSON.

RefineCo, Inc. Listed in the 2014 Branham300’s Next 50 Canadian Information and Communication Technology Companies

April 22, 2014

Results recognize RefineCo as one of Canada’s top technology companies.

RefineCo, an Alberta-based technology firm, is proud to announce their debut on the Branham300. They are now deemed to be one of “The Next 50 Canadian ICT Companies” to keep an eye out for.

“This award is validation of our approach towards achieving our vision to become the premier provider of SharePoint-based technology services – beginning in Alberta and extending from there. This wouldn’t be possible without our loyal client base and first-class employees,” says Eric Veenendaal, RefineCo’s CEO.

Founded in 2011 by two entrepreneurs, RefineCo has a proven track record of technical, operational and strategic management success. RefineCo has a strong and growing partnership with all of their customers as they continue to support them month-over-month by servicing their various technological needs.

This recognition reinforces the tremendous growth that RefineCo has experienced over the last year. RefineCo’s staff count and revenues have both doubled in size from 2012 to 2013, with similar growth expected in 2014.

About Branham300

The Branham300 highlights Canada’s top Information and Communication Technology (ICT) companies, as ranked by revenues. This recognition illustrates the depth and breadth of innovative technologies developed in Canada and is a leading source of intelligence on Canada’s ICT industry.

The Importance of Quality Assurance

April 11, 2014

When a stunning software, system or application is released to the public, people only show their admiration to the creators and developers. Admittedly, without their creation and development, the public wouldn’t have the chance to see the masterpiece. However, you should know that quality assurance (QA) also plays a rather important role.  Without QA, the public wouldn’t be able to experience the full potential of these masterpieces.

Why is QA important? Take software as an example. Human beings are fallible by nature and may make an error that produces a defect (fault, bug) in the program code. If the defect is executed, the software may fail to do what it should do, and the clients are not happy. Also, the time pressure, complex infrastructure, and system interactions may occur defects. QA is the key to identify, reduce and eliminate these defects in every stage of the software development life cycle. If the defects are dealt with as early as possible, the costs for development, test, and maintenance can be reduced dramatically.

Now, how to conduct QA? A common perception of QA is that it only consists of executing the software, but that is part of QA, not all. QA includes planning and control, analysis and design, implementation and execution, evaluating exit criteria and reporting, and closure activities. Take analysis and design for example. As a QA Analyst, I need to analyze the business requirements first, and then create detailed test cases. The business requirement may only say “a user must be able to create an account”, but I need to take one step further to determine all the situations. Maybe the e-mail address the user entered is malformed, or maybe the password doesn’t match to the 6-character standard. The coverage of the test cases is crucial. QA process may be time-consuming, but the potential time saved is much more than the time spent on QA. Clients would prefer to wait slightly longer for a high-quality product than receive a potentially flawed one sooner.

To be a good QA Analyst, the most important thing is to pay attention to details. You must be able to find things that others can’t by focusing on every detail of what you are testing. After identifying the defects or bugs, you need to report the issues to project managers and software developers through an issue or bug tracking system. Here at RefineCo, we use JIRA. Once an issue is created, we can see the time a bug was reported, the person who reported it, the developer who is working on fixing it, its severity, the erroneous program behavior, and the most important one, details on how to reproduce the bug. JIRA supports the life cycle of a bug through tracked status: To Do, Blocked, In Progress, Peer Review, Ready to Test, QA Approved, Done, and Closed. It provides a clear centralized overview of the bugs’ states and is convenient to use. The happiest thing is when all the bugs have been closed, which means we have high confidence in the software we created.

Remember, QA is the last chance to catch functional or other defects before the creation goes public or to clients. If you miss something, the public and clients will find it sooner or later. QA is not to be taken lightly and it plays a crucial role in software development.

Git 101

April 2, 2014

Here is a quick how-to on how to create a local Git repository and push to a remote repository.

For those that don’t know, Git is a source code management system that was initially developed by Linus Torvalds for the Linux Kernel.

First, open a new console session and check to see that Git is installed on your system.

$ git --version
git version

If Git is not installed on your system, please see

Once you have verified that Git is installed, the next step is to navigate to the root folder of your project and initialize a new repository.

$ git init
Initialized empty Git repository in d:/Development/ArchConnect/.git

Now that we have an empty Git repository we will want to add our file. First, we can check to see if there any files to add.

$ git status

On branch master

Initial commit

Untracked files:
(use "git add <file>..." to include in what will be committed)


nothing added to commit but untracked files present (use "git add" to track)

As you can see there are a number of untracked files so go ahead and add them to the repository.

$ git add .

Run the git status command again to see what Git has done.

$ git status
On branch master

Initial commit

Changes to be committed:
(use "git rm --cached <file>..." to unstage)

new file:   ArchConnect.Web/App_Start/RouteConfig.cs
new file:   ArchConnect.Web/ArchConnect.Web.csproj
new file:   ArchConnect.Web/ArchConnect.Web.csproj.user
new file:   ArchConnect.Web/Controllers/HomeController.cs
new file:   ArchConnect.Web/Global.asax
new file:   ArchConnect.Web/Global.asax.cs
new file:   ArchConnect.Web/Properties/AssemblyInfo.cs
new file:   ArchConnect.Web/Scripts/jquery-1.10.2.intellisense.js
new file:   ArchConnect.Web/Scripts/jquery-1.10.2.js

Next, commit the files to the local Git repository.

$ git commit -m "Initial Commit"
[master (root-commit) 855a778] Initial Commit

80 files changed, 79239 insertions(+)
create mode 100644 ArchConnect.Web/App_Start/RouteConfig.cs
create mode 100644 ArchConnect.Web/ArchConnect.Web.csproj
create mode 100644 ArchConnect.Web/ArchConnect.Web.csproj.user
create mode 100644 ArchConnect.Web/Controllers/HomeController.cs
create mode 100644 ArchConnect.Web/Global.asax
create mode 100644 ArchConnect.Web/Global.asax.cs
create mode 100644 ArchConnect.Web/Properties/AssemblyInfo.cs
create mode 100644 ArchConnect.Web/Scripts/jquery-1.10.2.intellisense.js
create mode 100644 ArchConnect.Web/Scripts/jquery-1.10.2.js

Now that all your code has been committed to your local repository, you will need to push your changes to a remote repository to keep them safe.

Usually your remote Git repository will be hosted on a server. The process of getting Git installed on a server is the same as on your workstation. (Alternatively, you can also create a GitHib account to host your remote Git repositories.)

Before you can push your code to your remote repository, you need to initialize a bare repository on the server.

$ git init ArchConnect.git --bare
Initialized empty Git repository in /home/ckooiker/git/repos/ArchConnect.git/

Back on your workstation add the remote repository.

$ git remote add origin username@myawesomeserver:/home/ckooiker/git/repos/ArchConnect.git

When you have your remote repository setup you can go ahead and push the changes.

$ git push origin master
Counting objects: 87, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (74/74), done.
Writing objects: 100% (87/87), 1.77 MiB | 0 bytes/s, done.
Total 87 (delta 3), reused 0 (delta 0)
To username@myawesomeserver:/home/ckooiker/git/repos/ArchConnect.git
* [new branch]      master -> master

And that’s it, our source code has been committed to remote repository!

For more information on Git and what it can all do checkout