Searching Multi-Node Tree Picker data (or any collection) with Examine

by Aaron Powell 22. September 2010 11:10

With the release of uComponents recently a lot of people are starting to work with a new data type called the MultiNodeTreePicker, and with this I’ve seen a few questions around searching the data it generates using Examine.

The problem is there is a catch, if you’re using the CSV storage (which you must if you’re working with Examine) you’ll hit a problem, the Examine index will have something like this:

1011,1231,1232,1225

But how do you search on that? Searching for ‘1231’ will not return anything, because it’s prefixed with ‘,’ and postfixed with ‘,’. So this brings a problem, how do you search?

Bring on Events

As Shannon spoke about at CodeGarden 10 Examine has a number of different events you can hook into to do different things (slides and code) and this is what we’re going to need to work with.

I’ve touched on events before but this time we’re going to look at a different event, we’re going to look at the GatheringNodeData event.

GatheringNodeData event

So this event in Examine is fired while Examine is scraping the data out of an XML element which it has received. This XML could be from Umbraco (in the scenario we’re looking at here) or it could be from your own data source, and the event is raised once Examine as turned the XML into a Key/ Value representation of it.

The event that raises has custom event arguments, which has a property called Fields. This Fields property is a dictionary which contains the full Key/ Value representation of the data which will end up in Examine!

Now this dictionary is able to be manipulated, so you can add/ remove data as you see if (but that’s a topic for another blog), it also means you can change the data!

Changing the data for our needs

As I mentioned at the start of this we end up with comma-separated string from the datatype which isn’t useful for searching, so we can use an event handler to change what we’ve got. First we need to tie an event handler

public class ExamineEvents : ApplicationBase 
{
	public ExamineEvents() 
	{
		var indexer = ExamineManager.Instance.IndexProviderCollection["MyIndexer"];
		indexer.GatheringNodeData += new EventHandler(GatheringNodeDataHandler);
	}

	void GatheringNodeDataHandler(object sender, IndexingNodeDataEventArgs e)
	{
		//do stuff here
	}
}

So this is just a simple wire-up, using the ApplicationBase class in Umbraco so that it’ll be created on application start-up. Next we need to implement the event handler:

void GatheringNodeDataHandler(object sender, IndexingNodeDataEventArgs e)
{
	//grab the current data from the Fields collection
	var mntp = e.Fields["TreePicker"];
	//let's get rid of those commas!
	mntp = mntp.Replace(",", " ");
	//now put it back into the Fields so we can pretend nothing happened!
	e.Fields["TreePicker"] = mntp;
}

And you’re done! Now the data will be written into the index with spaces rather than commas meaning that you can search on each ID without the need for wildcards or any other “hacks” to get it to work.

Note: This will work in the majority of cases, the only reason it’ll fail is if you’re using an analyzer that strips out numbers before indexing. For more information about Lucene analyzers take a look at this article: http://www.aaron-powell.com/lucene-analyzer

VisualSVN server on SVN protocol

by Shannon Deminick 20. September 2010 04:53

I’m sure I’m not the only one who has noticed that running SVN over the Http protocol using VisualSVN is REALLY slow in comparison to running SVN using the file:/// or svn:// protocol. It is nice having the option of the http protocol so at least you can browse your repositories in your browser, allow external access to them without opening up another port on your firewall and also apply Windows security to your repositories, however, it is really, really slow. After some Googling on how to get VisualSVN server to run using the SVN protocol, it turns out this is not possible but you can run the SVN protocol as a service in tandem with VisualSVN which will give you the best of both worlds. Luckily for us, VisualSVN installs all of the necessary files for us to do this. Here’s how:

  • Create a batch file in your VisualSVN bin folder (normally: C:\Program Files\VisualSVN Server\bin) called something like: “INSTALLSVNPROTOCOL.bat”
    • You’ll need to edit the below script to map your svn repository folders properly. Change the “E:\YOUR-SVN-REPOSITORY-ROOT-FOLDER” to the path of your svn repository root folder.
echo ---Install the service REM this should all be on one line! sc create SVNPROTOCOLSERVICE binpath= "\"c:\Program Files\VisualSVN Server\bin\svnserve.exe\" --service --root \"E:\YOUR-SVN-REPOSITORY-ROOT-FOLDER\" " displayname= "SVN Service" depend= Tcpip echo ---Config to auto-start sc config SVNPROTOCOLSERVICE start= auto
  • Next, run your batch file.
    • This will install a windows service to host your repositories on the SVN protocol
  • Update your windows service to run as Administrator, or a user that has the permissions to run the service
    • Start Menu –> Adminstrative Tools –> Services –> Find the “SVN Service” that was just created –> Right click –> Properties –> Log On  Tab –> Change “Log on as:” to use your Administrator account.
  • Start the windows service

Your done! You can now access your repositories via the SVN protocol using something like:

svn://yourservername.yourdomainname.local/YOUR-REPOSITORY-NAME

 

Ok, to uninstall:

  • Create a batch file in the same folder as your install batch file called something like “UNINSTALLSVNPROTOCOL.bat”
echo --remove svn service sc stop SVNPROTOCOLSERVICE sc delete SVNPROTOCOLSERVICE
  • Run the batch file
Tags: ,

Automated website deployment with PowerShell and SmartFTP

by Shannon Deminick 2. September 2010 07:56

SmartFTP is a fantastic FTP application which handles syncing files very effectively. This means that when you upload your entire website, SmartFTP will automatically detect changes and only upload what is required (instead of overwriting all of the files like some FTP applications do). For each project at TheFARM we have build scripts which run and create a time stamped ZIP package for each deployment environment with all of the necessary files formatted appropriately for each. Our deployment process then involves unzipping the contents of this file, opening up SmartFTP, connecting to the deployment destination and transfering all of the deployment files up (which SmartFTP synchronizes for us).

I thought it would be much more efficient if we automated this process. So we did some investigation and it turns out the SmartFTP conveniently has an API! So we decided to see if we could write a PowerShell script to use the SmartFTP api to automagically transfer/sync all of our deployment files in our Zip package to the necessary FTP site and with a bit of trial and error we managed to do it! Now, I’m not PowerShell expert or anything, and in fact this was my very first PowerShell script ever written so I’m sure this could all be done a bit better, but it works! I’m not going to go into detail about the SmartFTP api or how to write PowerShell stuff because this script will work with some basic requirements:

  • You need both PowerShell and SmartFTP installed
  • Currently this only supports the standard FTP protocol, but if you need SFTP, etc… you can just change the $fav variable’s ‘Protocol’ property
  • The parameters, in this order are:
    • destination
      • the IP address, or host of your FTP server
    • user
      • the username used to login to the FTP server
    • password
      • the password used to login to the FTP server
    • path
      • The FTP path of where you want your files to go on your FTP server
    • port
      • The FTP port to use, default is 21
    • source
      • The source folder to copy to the FTP site, if not specified, uses the current directory that the PowerShell script is run from

Example usage:

FTPSync.ps1 123.123.123.123 MyUserName MyPassword 21 “C:\MyWebsiteFolder” “/websites/MyWebsite”

or you can just double click on the ps1 file and it will prompt you for these details.

So without further adieu, here’s the script!

#requires -version 2.0 # Define inputs param ( [parameter(Mandatory=$true)] [string] $dest, [parameter(Mandatory=$true)] [string] $user, [parameter(Mandatory=$true)] [string] $pass, [parameter(Mandatory=$true)] [ValidatePattern('\d+')] [int] $port = 21, [parameter(Mandatory=$false)] [ValidateScript({ Test-Path -Path $_ -PathType Container })] [string] $source, [parameter(Mandatory=$true)] [ValidatePattern('\/+')] [string] $path ) # get current folder $currFolder = (Get-Location -PSProvider FileSystem).ProviderPath; # set current folder [Environment]::CurrentDirectory=$currFolder; # if the source isn't set, then use the current folder if ($source = "") { $source = $currFolder; } Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black Write-Host("{0, -20}{1,20}" -f "Destination", $dest); Write-Host("{0, -20}{1,20}" -f "User", $user); Write-Host("{0, -20}{1,20}" -f "Pass", "********"); Write-Host("{0, -20}{1,20}" -f "Port", $port); Write-Host ""; Write-Host "Source:"; Write-Host $source; Write-Host ""; Write-Host "Path"; Write-Host $path; Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black # Create application $smartFTP = New-Object -comObject SmartFTP.Application; $smartFTP.Visible = [bool]0; $smartFTP.CloseAll(); # create temp favorite item $fav = $smartFTP.CreateObject("sfFavorites.FavoriteItem"); $fav.Name = $user + " @ " + $dest + " (temp favorite by cmdInterface)"; # 1 = FTP standard protocol $fav.Protocol = 1; $fav.Host = $dest; $fav.Port = $port; $fav.Path = $path; $fav.Username = $user; $fav.Password = $pass; # forces it not to be saved $fav.Virtual = "true"; # Add temporary favorite to SmartFTPs FavoriteManager $favMgr = $smartFTP.FavoritesManager; $rootFolder = $favMgr.RootFolder; $rootFolder.AddItem($fav); # Get the transfer queue $queue = $smartFTP.TransferQueue; # stop the queue if it isn't already if ($queue.State -ne 1) { $queue.Stop(); } # Stopped = 1 # clear the queue foreach($item in $queue.Items) { $queue.RemoveItem($item); } # set the thread count for the queue $queue.MaxWorkers = 20; #enable logging $queue.Log = "true"; $queue.LogFolder = $currFolder + "\\LOG"; # create new transfer item $newItem = $smartFTP.CreateObject("sfTransferQueue.TransferQueueItem"); # set the item as a folder and copy operation, $newItem.type = 2; #FOLDER = 2 $newItem.Operation = 1; #COPY = 1 # Set the source $newItem.Source.type = 1; #LOCAL = 1 $newItem.Source.Path = $source; # Set the destination $newItem.Destination.type = 2; #REMOTE = 2 $newItem.Destination.Path = $path; $newItem.Destination.FavoriteIdAsString = $fav.IdAsString; #links up to our connection favorite # and finally add it $queue.AddItemTail($newItem); Write-Host "STARTING" -foregroundcolor yellow -backgroundcolor black; $queue.Start(); while ($queue.Items.Count -ne 0) { Write-Host "Processing...bytes transfered: " $queue.TransferredBytes; Start-Sleep -s 2; #wait 2 seconds } # store the total bytes $totalBytes = $queue.TransferredBytes; # cleanup smartftp app $queue.Quit(); $smartFTP.Exit(); # parse logs # regex to find "[DATE/TIME] STOR FILENAME # which indicates a file transfer $regex = new-object System.Text.RegularExpressions.Regex("\[[\w\-\:]*?\]\sSTOR\s(.+?)\[",,[System.Text.RegularExpressions.RegexOptions]::SingleLine); $totalFiles = 0; Write-Host "Files Transfered" -foregroundcolor cyan -backgroundcolor black Get-ChildItem $queue.LogFolder -include *.log -Recurse | foreach ($_) { $currFile = Get-Content $_.fullname; $match = $regex.Matches($currFile); if ($match.Count -gt 0) { foreach($m in $match) { Write-Host $m.Groups[1]; } $totalFiles++; } remove-item $_.fullname -Force -Recurse ; } Write-Host "COMPLETED (total bytes: " $totalBytes ", total files: )" $totalFiles -foregroundcolor cyan -backgroundcolor black; "------------------------------------------------------" # cleanup COM Remove-Variable smartFTP