Searching Umbraco using Razor and Examine

by Shannon Deminick 15. March 2011 05:51

Since Razor is really just c# it’s super simple to run a search in Umbraco using Razor and Examine.  In MVC the actual searching should be left up to the controller to give the search results to your view, but in Umbraco 4.6 + , Razor is used as macros which actually ‘do stuff’. Here’s how incredibly simple it is to do a search:

@using Examine; @* Get the search term from query string *@ @{var searchTerm = Request.QueryString["search"];} <ul class="search-results"> @foreach (var result in ExamineManager.Instance.Search(searchTerm, true)) { <li> <span>@result.Score</span> <a href="@umbraco.library.NiceUrl(result.Id)"> @result.Fields["nodeName"] </a> </li> } </ul>

That’s it! Pretty darn easy.

And for all you sceptics who think there’s too much configuration involved to setup Examine, configuring Examine requires 3 lines of code. Yes its true, 3 lines, that’s it. Here’s the bare minimum setup:

1. Create an indexer under the ExamineIndexProviders section:

<add name="MyIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"/>

2. Create a searcher under the ExamineSearchProviders section:

<add name="MySearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"/>

3. Create an index set under the ExamineLuceneIndexSets config section:

<IndexSet SetName="MyIndexSet" IndexPath="~/App_Data/TEMP/MyIndex" />

This will index all of your data in Umbraco and allow you to search against all of it. If you want to search on specific subsets, you can use the FluentAPI to search and of course if you want to modify your index, there’s much more you can do with the config if you like.

With Examine the sky is the limit, you can have an incredibly simple index and search mechanism up to an incredibly complex index with event handlers, etc… and a very complex search with fuzzy logic, proximity searches, etc…  And no matter what flavour you choose it is guaranteed to be VERY fast and doesn’t matter how much data you’re searching against.

I also STRONGLY advise you to use the latest release on CodePlex: http://examine.codeplex.com/releases/view/50781 . There will also be version 1.1 coming out very soon.

Enjoy!

Categories: Umbraco | Examine | .Net

Consume Json REST service with WCF and dynamic object response

by Shannon Deminick 11. February 2011 06:26

This was my first shot at WCF and though I saw how nice it could be, i failed miserably… well, I’m actually not too sure. I tried all sorts of things but no matter what couldn’t get WCF to behave and return the actual response you would get if you just consumed the service with the WebClient object. Here’s what I tried and assumed that this ‘should’ work but the result was always an empty dictionary:

Note: this demo is based off of the @Task project management tool’s REST API.

Define the contract:

[ServiceContract] public interface IAtTaskService { [OperationContract] [WebGet( UriTemplate = "/api/login?username={username}&password={password}", RequestFormat = WebMessageFormat.Json)] IDictionary<string, object> Login(string username, string password); }

Run the code:

public IDictionary<string, object> DoThis(string username, string pword) { using (var cf = new WebChannelFactory<IAtTaskService>( new Uri("https://mycompaniesurl.attask-ondemand.com/attask/"))) { var s = cf.CreateChannel(); var token = s.Login(username, pword); return token; } }

The above seemed really nice to be able to consume services but the result was always empty. I tried using strongly typed objects in the result, made sure it wasn’t an HTTPS issue, used the normal WCF configuration approach, etc… but couldn’t get it to work.

The ‘Fix’

I quite like the way that WCF has UriTemplates and you can just attribute up an interface and then make strongly typed calls to access your services so i decided I’d just make that work with my own implementation. Again, perhaps I was ‘doing it wrong’ or whatever, but the below implementation is pretty cool and also lets you get JSON results as a dynamic object :)

Define the contract:

This is pretty much the same as above, but we have a dynamic response

[ServiceContract] public interface IAtTaskService { [OperationContract] [WebGet( UriTemplate = "/api/login?username={username}&password={password}", RequestFormat = WebMessageFormat.Json)] dynamic Login(string username, string password); }

The JsonRestService base class

I created a class that lets you specify you contract service as a generic argument and then provides you with a method called “Send” which is defined a:

protected dynamic Send(Func<T, dynamic> func)

This class will manage all of the http requests and use the WCF attributes applied to your contract to generate the correct URLs and parameters (source code at the bottom of this article)

Define a standard service

So to consume the REST services, we just create a ‘real’ service class such as:

public class AtTaskService : JsonRestService<IAtTaskService>, IAtTaskService { public AtTaskService(string serviceUrl) : base(serviceUrl) { } public dynamic Login(string username, string password) { return Send(x => x.Login(username, password)); } }

As you can see, the “Login” method just calls the underlying “Send” method which uses a strongly typed delegate. The Send method then inspects the delegate and generates the correct URL with parameters based on the WCF attributes.

Using the service

So now, to use the new service we can do something like:

[TestMethod] public void AtAtask_Login() { //Arrange var svc = new AtTaskService("https://mycompanyname.attask-ondemand.com/attask/"); //Act var token = svc.Login("MyUserName", "mypassword"); //Assert Assert.IsTrue(token != null); Assert.IsTrue(token.data != null); Assert.IsTrue(token.data.userID != null); }

Too easy!

As I mentioned earlier, I would have assumed that WCF should handle this out of the box, but for the life of my I couldn’t get it to return any results. Whether it was a request issue or a parsing issue, or whatever i have no idea. In the meantime, here’s the source code for the JsonRestService class which will allow you to do the above. The source requires references to Json.Net and Aaron-powell.dynamics

JsonRestService source

Disclaimer: I made this today in a couple of hours, I’m sure there’s some code in here that could be refactored to be nicer

public class JsonRestService<T> { public string ServiceUrl { get; private set; } public JsonRestService(string serviceUrl) { ServiceUrl = serviceUrl; } /// <summary> /// Queries the WCF attributes of the method being called, builds the REST URL and sends the http request /// based on the WCF attribute parameters. When the JSON response is returned, it is changed to a dynamic object. /// </summary> /// <param name="func"></param> /// <returns></returns> protected dynamic Send(Func<T, dynamic> func) { //find the method on the main type that is being called... i'm sure there's a better way to do this, //but this does work. //This will not work if there are methods with overloads. var methodNameMatch = Regex.Match(func.Method.Name, "<(.*?)>"); if (!methodNameMatch.Success && methodNameMatch.Groups.Count == 2) { throw new MissingMethodException("Could not find method " + func.Method.Name); } var realMethodName = methodNameMatch.Groups[1].Value; var m = typeof(T).GetMethods().Where(x => x.Name == realMethodName).SingleOrDefault(); if (m == null) { throw new MissingMethodException("Could not find method" + realMethodName + " on type " + typeof(T).FullName); } //now that we have the method, find the wcf attributes var a = m.GetCustomAttributes(false); var webGet = a.OfType<WebGetAttribute>().SingleOrDefault(); var webInvoke = a.OfType<WebInvokeAttribute>().SingleOrDefault(); var httpMethod = webGet != null ? "GET" : webInvoke != null ? webInvoke.Method : string.Empty; if (string.IsNullOrEmpty(httpMethod)) { throw new ArgumentNullException("The WebGet or WebInvoke attribute is missing from the method " + realMethodName); } //now that we have the WCF attributes, build the REST url based on the url template and the //method being called with it's parameters var urlTemplate = webGet != null ? webGet.UriTemplate : webInvoke.UriTemplate; var urlWithParams = GetUrlWithParams(urlTemplate, func); var url = ServiceUrl + urlWithParams; //Do the web requests string output; if (httpMethod == "GET") { output = HttpGet(url); } else { //need to move the query string params to the http parameters var parts = url.Split('?'); output = HttpInvoke(parts[0], httpMethod, parts.Length > 1 ? parts[1] : string.Empty); } //change the response to json and then dynamic var stringReader = new StringReader(output); var jReader = new JsonTextReader(stringReader); var jsonSerializer = new JsonSerializer(); var json = jsonSerializer.Deserialize<Dictionary<string, object>>(jReader); return json.AsDynamic(); } /// <summary> /// Updates the url template with the correct parameters /// </summary> /// <param name="template"></param> /// <param name="expression"></param> /// <returns></returns> private static string GetUrlWithParams(string template, Func<T, dynamic> expression) { //parse the template, get the matches foreach (var m in Regex.Matches(template, @"\{(.*?)\}").Cast<Match>()) { if (m.Groups.Count == 2) { var m1 = m; //find the fields based on the expression(Func<T>), get their values and replace the tokens in the url template var field = expression.Target.GetType().GetFields().Where(x => x.Name == m1.Groups[1].Value).Single(); template = template.Replace(m.Groups[0].Value, field.GetValue(expression.Target).ToString()); } } return template; } /// <summary> /// Do an Http GET /// </summary> /// <param name="uri"></param> /// <returns></returns> private static string HttpGet(string uri) { var webClient = new WebClient(); var data = webClient.DownloadString(uri); return data; } /// <summary> /// Do an Http POST/PUT/etc... /// </summary> /// <param name="uri"></param> /// <param name="method"></param> /// <param name="parameters"></param> /// <returns></returns> private static string HttpInvoke(string uri, string method, string parameters) { var webClient = new WebClient(); var data = webClient.UploadString(uri, method, parameters); return data; } }
Tags: , ,
Categories: .Net

2011 Umbraco Sydney Meetup

by Shannon Deminick 18. January 2011 10:20

TheFARM is proud to host another Umbraco meetup. With the recent release of Juno (4.6) and the current work being done on Jupiter (5.0), there will be plenty to discuss! 3 of the Umbraco core developers will be attending (Shannon Deminick, Aaron Powell & Pete Gregory) so there will be a lot of knowledge to share.

As always, TheFARM will be supplying the venue, beer & snacks. Please register on the Our Umbraco site here: http://our.umbraco.org/events/sydney-festival-of-umbraco. If you’re interested in Umbraco, then you should definitely come along!

Event details:

Address: Suite 101/4-14 Buckingham Street, SURRY HILLS, NSW
Date: February 1st 2011
Time: 6 PM

Tags:
Categories:

Binding a custom property with model binders

by Shannon Deminick 18. January 2011 09:00

In ASP.Net MVC Model Binders are great but what if you have a property on your model that is a ‘Simple’ type that you want to assign to a custom model binder? Take the following class for example:

public class MyModel { [Required] public string Name { get; set; } [Required] public DateTime StartDate { get; set; } [Required] public DateTime StartTime { get; set; } [CustomValidation(typeof(CreateJourneyModel), "OnDateValidate")] public DateTime FullStartDateTime { get; set; } //Custom validation logic which needs to validate the full date/time internal static ValidationResult OnDateValidate(DateTime dt, ValidationContext validationContext) { return dt < DateTime.Now.AddHours(1) ? new ValidationResult("Enter a date/time greater than 3 hours from now") : ValidationResult.Success; } }

The above model will allow us to have a different date and time field rendered for input on a form but requires the combined date/time to run custom validation against. You might already be thinking: Isn’t this the same problem that Scott Hanselman solved in this post? Well, sort of but not really. What Scott did was assign a custom model binder to all instances of DateTime or set specifically on an Action parameter and was also splitting one DateTime field into 2 on the front-end . I’m not really a fan of having a custom model binder for all instances of DateTime in my application , I don’t have a DateTime parameter in my Action to explicitly assign a model binder too and I also decided to use 2 DateTime properties to represent 2 different fields on my form.  The solution presented below could allow you to custom bind any single property of your model which you could then assign custom validation to.

First, we need to put the fields on your form so they get posted back to the server. Using the Html.EditorFor(x => …..) syntax is the way to go, however, for the field we want to custom bind using Html.HiddenFor(x => …. ) is better so the field isn’t actually displayed on the form. This will add this field name to the ValueProvider when it’s posted to the server.

Next, create the model binder. Since we cannot assign a model binder to a property of a model, we need to make the binder for the model containing the property. And, since we don’t want to do any custom binding for the rest of the properties, we’ll just inherit from DefaultModelBinder and let it do the work for us. The only thing we have to do is override GetPropertyValue , check for the property that we want to custom bind and bind it!

public class MyCustomModelBinder : DefaultModelBinder { protected override object GetPropertyValue(ControllerContext controllerContext, ModelBindingContext bindingContext, System.ComponentModel.PropertyDescriptor propertyDescriptor, IModelBinder propertyBinder) { //check if it the 'FullStartDateTime' property if (propertyDescriptor.Name == "FullStartDateTime") { //try to get the date part var dateValue = bindingContext.ValueProvider.GetValue("StartDate"); DateTime date; if (!string.IsNullOrEmpty(dateValue.AttemptedValue) && DateTime.TryParseExact(dateValue.AttemptedValue, "dd/MM/yyyy", CultureInfo.InvariantCulture, DateTimeStyles.None, out date)) { //try to get the time part var timeValue = bindingContext.ValueProvider.GetValue("StartTime"); DateTime time; if (!string.IsNullOrEmpty(timeValue.AttemptedValue) && DateTime.TryParseExact(timeValue.AttemptedValue, "HH:mm", CultureInfo.InvariantCulture, DateTimeStyles.None, out time)) { //we've got both parts, join them together var fullDateTime = new DateTime(date.Year, date.Month, date.Day, time.Hour, time.Minute, time.Second); return fullDateTime; } } //could not bind return null; } //let the default model binder do it's thing return base.GetPropertyValue(controllerContext, bindingContext, propertyDescriptor, propertyBinder); } }

Last, all we need to do is assign the custom model binder to the custom model which can easily be done by just attributing the model:

[ModelBinder(typeof(MyCustomModelBinder))] public class MyModel {...}

That's it! Now all validation is handled as per normal on the custom property.

Tags:
Categories: .Net

Importing SVN to Mercurial with complex SVN repository

by Shannon Deminick 2. November 2010 05:14

Here @ TheFARM, we’ve been moving across to Mercurial (on BitBucket) for our code repositories. In many cases our SVN repositories are structured ‘normally’:

  • trunk
  • tags
  • branches

Using the ‘hg convert’ command line, when your SVN repository is structured this way will import your trunk into the Mercurial ‘default’ branch and your branches/tags into named branches. This also imports all history and revisions. From there, you can merge as you wish to structure your Mercurial repository the way that you want.

However, in some cases we have more complicated repositories. An example of this is a structure like the following:

  • trunk
    • DotNet
    • Flash
  • tags
  • branches
    • v1.1-DotNet
    • v1.2-Flash

In the above structure, we’ve actually branched the trunk/DotNet & trunk/Flash folders separately into their own branches. Unfortunately, Mercurial doesn’t operate this way so it doesn’t really understand creating branches from folders. There’s a couple different ways that you can get this from SVN into Mercurial whilst maintaining all of your history…

One way is to run ‘hg convert’ on the entire repository. You’ll end up with 3 branches in Mercurial: default, v1.1-DotNet & v1.2-Flash. The problem is that if you try to merge the named branches into default, you’ll end up with a mess since the branches don’t have the same folder structure as default. To overcome this, you can restructure each named branch to follow the same folder structure as default. To do this, we us the ‘rename’ method on Tortoise Hg. So for instance, if we had this folder structure inside of v1.1-DotNet:

  • BuildFiles
  • MyProject.Web
  • MyProject.Config

So that we can merge this with default we need to restructure this into:

  • DotNet
    • BuildFiles
    • MyProject.Web
    • MyProject.Config

So we just need to right click each folder seperately, and select the rename option from the Tortoise Hg sub menu:

image

Then we prefix the folder name with the new folder location which will the ‘move’ the file:

image

Now that the named branch v1.1-DotNet is in the same folder structure as default, we can perform a merge.

The other way to import a complicated SVN structure to mercurial is to convert individual branches to mercurial repositories one by one. The first thing you’ll need to do is run an ‘hg convert’ on the Trunk of your SVN repository. This will create your new ‘master’ mercurial repository for which will push the other individual mercurial repositories in to. Next, run an ‘hg convert’ on each of your SVN branches. For example: hg convert svn://my.svn.server.local/MyProject/Branches/v1.1-DotNet.

Once you have individual repositories for your branches, we can force push these into your ‘master’ repository. To do a merge of these branches, the above procedure will still need to be followed to ensure your branches have the same folder structure as default. HOWEVER, because we’ve forced pushed changesets into Mercurial, it has no idea how these branches relate to each other (in fact, it gives you warnings about this when you force push). When you try to do a merge, you’ll end up getting conflict warnings for every file that exists in both locations since Mercurial doesn’t know which one is newer/older. This can be a huge pain in the arse, especially if you have tons of files. If we assume that the branch files are the most up to date and we just want to replace the files in default, then there’s a fairly obscure way to do that. In the merge dialog, you’ll need to select the option “internal : other” from the list of Merge tools:

image

This tells Mercurial that for any conflict you want to use the ‘other’ revision (which is your branch revision since you should have default checked out to do the merge).

We’ve had success with both of these options for converting SVN to Mercurial and maintaining our history.

Categories: Source Control

Examine output indexing

by Shannon Deminick 1. November 2010 15:39

Last week Pete Gregory (@pgregorynz) and I were discussing different implementations of Examine. Particularly when you need to use Examine events to collate information from different nodes to put into the index for the page being rendered. An example of this is an FAQ engine where you might have an Umbraco content structure such as:

  • Site Container
    • Public
      • FAQs
        • FAQ Item 1
        • FAQ Item 2
        • FAQ Item 3

In this example, the page that is rendered to the end user is FAQs but the data from all 4 nodes (FAQs, FAQ Item 1 –> 4) needs to be added to the index for the FAQs page. To do this you can use Examine events, either using the GatheringNodeData of the BaseIndexProvider, or by using the DocumentWriting event of the UmbracoContentIndexer (I’ll write another post covering the difference between these two events and why they both exist). Though writing Examine event handlers to put the data from FAQ Item 1 –> 4 into the FAQs index isn’t very difficult, it would still be really cool if all of this could be done automatically.

Pete mentioned it would be cool if we could just index the output html of a page (sort of like Google) and suddenly the ideas started to flow. This concept is actually quite easy to do so within the next month or so we’ll probably release a beta of Examine Output Indexing. Here’s the way it’ll all get put together:

  • An HttpModule will be created to do 2 things:
    • Check if the current request is an Umbraco page request
      • If it is, we can easily get the current node being rendered since it’s already been added to the HttpContext items by Umbraco
      • Use the standard Examine handlers to enter the node’s data into the indexes based on the configuration you’ve specified in your Examine configuration files
    • Get the HTML output of the page before it is rendered to the end user, parse the html to get the relevant data and put it into the index for the current Umbraco page
  • We figured that it would also be cool to have an Examine node property that developers could defined called something like: examineNoIndex which we could check for when we determine that it’s an Umbraco page and if this property is set to true, we’ll not index this page.
    • This could give developers more control over what specific pages shouldn’t be indexed based directly from the CMS properties instead of writing custom events

With the above, a developer will simply need to put the HttpModule in their web.config, define an Examine index based on a new provider we create and that’s it. There will be no need to manually collate node data such as the above FAQ example. However, please note that this will work for straight forward searching so if you have complex searching & indexing requirements, I would still recommend using events since you have far more control over what information is indexed.

Any feedback is much appreciated since we haven’t started developing this quite yet.

Categories: Examine | Umbraco | .Net

Examine v1.0 RTM

by Shannon Deminick 22. October 2010 04:46

We finally released Examine version 1.0 a week or so ago. You can find the latest download package from the CodePlex downloads page for Examine: http://examine.codeplex.com/releases/view/50781 

Here’s what you’ll need to know

  • There are some breaking changes from the version that is shipped with Umbraco 4.5 and also from the Examine RC3 release. The downloads tab on CodePlex contains the Release Notes for download which contains all of the information on upgrading & breaking changes
  • READ THE RELEASE NOTES BEFORE UPGRADING
  • There’s a ton of bugs fixed in this release from the version shipped with Umbraco 4.5
  • Lots of new features have been added:
    • Indexing ANY type of data easily using the LuceneEngine index/search providers
    • PDF Indexing for Umbraco
    • XSLT extensions for Umbraco
    • Data Type declarations for indexed fields
    • Date & Number range searching
  • New documentation has been added to CodePlex

Using v1.0 RTM on Umbraco 4.5

The upgrade process from the Examine version shipped with 4.5 to v1.0 RTM should be pretty seamless (unless you are using some specific API calls as noted in the release notes). However, once you drop in the new DLLs you’ll probably notice that the internal search no longer works. This is due to a bug in the Umbraco 4.5. codebase and an non-optimal implementation of Examine which has to do with case sensitivity for application aliases (i.e. Content vs content ). The work-around is simple though: all we need to do is change the Analyzer used for the internal searcher in the Examine configuration file to use the StandardAnalyzer instead of the WhitespaceAnalyzer. This is because the WhitespaceAnalyzer is case sensitive whereas the StandardAnalyzer is not. This issue is fixed in Umbraco Juno (4.6) and will continue to use the WhitespaceAnalyzer so that Examine doesn’t tokenize strings that contain punctuation. For more info on Analyzers, have a look at Aaron’s post.

Next Versions

There probably won’t be too many more changes coming for Examine v1.0 apart from any bug fixing that needs to be done and maybe some tweaks to the Fluent API. We will start working on v2.0 at some point this year or early next year which will take Examine to the next level. It will be less focused on configuration, have a smaller foot print and be much more configurable through code (such as how ASP.Net MVC works).

Categories: Examine | Umbraco

VisualSVN server on SVN protocol

by Shannon Deminick 20. September 2010 04:53

I’m sure I’m not the only one who has noticed that running SVN over the Http protocol using VisualSVN is REALLY slow in comparison to running SVN using the file:/// or svn:// protocol. It is nice having the option of the http protocol so at least you can browse your repositories in your browser, allow external access to them without opening up another port on your firewall and also apply Windows security to your repositories, however, it is really, really slow. After some Googling on how to get VisualSVN server to run using the SVN protocol, it turns out this is not possible but you can run the SVN protocol as a service in tandem with VisualSVN which will give you the best of both worlds. Luckily for us, VisualSVN installs all of the necessary files for us to do this. Here’s how:

  • Create a batch file in your VisualSVN bin folder (normally: C:\Program Files\VisualSVN Server\bin) called something like: “INSTALLSVNPROTOCOL.bat”
    • You’ll need to edit the below script to map your svn repository folders properly. Change the “E:\YOUR-SVN-REPOSITORY-ROOT-FOLDER” to the path of your svn repository root folder.
echo ---Install the service REM this should all be on one line! sc create SVNPROTOCOLSERVICE binpath= "\"c:\Program Files\VisualSVN Server\bin\svnserve.exe\" --service --root \"E:\YOUR-SVN-REPOSITORY-ROOT-FOLDER\" " displayname= "SVN Service" depend= Tcpip echo ---Config to auto-start sc config SVNPROTOCOLSERVICE start= auto
  • Next, run your batch file.
    • This will install a windows service to host your repositories on the SVN protocol
  • Update your windows service to run as Administrator, or a user that has the permissions to run the service
    • Start Menu –> Adminstrative Tools –> Services –> Find the “SVN Service” that was just created –> Right click –> Properties –> Log On  Tab –> Change “Log on as:” to use your Administrator account.
  • Start the windows service

Your done! You can now access your repositories via the SVN protocol using something like:

svn://yourservername.yourdomainname.local/YOUR-REPOSITORY-NAME

 

Ok, to uninstall:

  • Create a batch file in the same folder as your install batch file called something like “UNINSTALLSVNPROTOCOL.bat”
echo --remove svn service sc stop SVNPROTOCOLSERVICE sc delete SVNPROTOCOLSERVICE
  • Run the batch file
Tags: ,

Automated website deployment with PowerShell and SmartFTP

by Shannon Deminick 2. September 2010 07:56

SmartFTP is a fantastic FTP application which handles syncing files very effectively. This means that when you upload your entire website, SmartFTP will automatically detect changes and only upload what is required (instead of overwriting all of the files like some FTP applications do). For each project at TheFARM we have build scripts which run and create a time stamped ZIP package for each deployment environment with all of the necessary files formatted appropriately for each. Our deployment process then involves unzipping the contents of this file, opening up SmartFTP, connecting to the deployment destination and transfering all of the deployment files up (which SmartFTP synchronizes for us).

I thought it would be much more efficient if we automated this process. So we did some investigation and it turns out the SmartFTP conveniently has an API! So we decided to see if we could write a PowerShell script to use the SmartFTP api to automagically transfer/sync all of our deployment files in our Zip package to the necessary FTP site and with a bit of trial and error we managed to do it! Now, I’m not PowerShell expert or anything, and in fact this was my very first PowerShell script ever written so I’m sure this could all be done a bit better, but it works! I’m not going to go into detail about the SmartFTP api or how to write PowerShell stuff because this script will work with some basic requirements:

  • You need both PowerShell and SmartFTP installed
  • Currently this only supports the standard FTP protocol, but if you need SFTP, etc… you can just change the $fav variable’s ‘Protocol’ property
  • The parameters, in this order are:
    • destination
      • the IP address, or host of your FTP server
    • user
      • the username used to login to the FTP server
    • password
      • the password used to login to the FTP server
    • path
      • The FTP path of where you want your files to go on your FTP server
    • port
      • The FTP port to use, default is 21
    • source
      • The source folder to copy to the FTP site, if not specified, uses the current directory that the PowerShell script is run from

Example usage:

FTPSync.ps1 123.123.123.123 MyUserName MyPassword 21 “C:\MyWebsiteFolder” “/websites/MyWebsite”

or you can just double click on the ps1 file and it will prompt you for these details.

So without further adieu, here’s the script!

#requires -version 2.0 # Define inputs param ( [parameter(Mandatory=$true)] [string] $dest, [parameter(Mandatory=$true)] [string] $user, [parameter(Mandatory=$true)] [string] $pass, [parameter(Mandatory=$true)] [ValidatePattern('\d+')] [int] $port = 21, [parameter(Mandatory=$false)] [ValidateScript({ Test-Path -Path $_ -PathType Container })] [string] $source, [parameter(Mandatory=$true)] [ValidatePattern('\/+')] [string] $path ) # get current folder $currFolder = (Get-Location -PSProvider FileSystem).ProviderPath; # set current folder [Environment]::CurrentDirectory=$currFolder; # if the source isn't set, then use the current folder if ($source = "") { $source = $currFolder; } Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black Write-Host("{0, -20}{1,20}" -f "Destination", $dest); Write-Host("{0, -20}{1,20}" -f "User", $user); Write-Host("{0, -20}{1,20}" -f "Pass", "********"); Write-Host("{0, -20}{1,20}" -f "Port", $port); Write-Host ""; Write-Host "Source:"; Write-Host $source; Write-Host ""; Write-Host "Path"; Write-Host $path; Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black # Create application $smartFTP = New-Object -comObject SmartFTP.Application; $smartFTP.Visible = [bool]0; $smartFTP.CloseAll(); # create temp favorite item $fav = $smartFTP.CreateObject("sfFavorites.FavoriteItem"); $fav.Name = $user + " @ " + $dest + " (temp favorite by cmdInterface)"; # 1 = FTP standard protocol $fav.Protocol = 1; $fav.Host = $dest; $fav.Port = $port; $fav.Path = $path; $fav.Username = $user; $fav.Password = $pass; # forces it not to be saved $fav.Virtual = "true"; # Add temporary favorite to SmartFTPs FavoriteManager $favMgr = $smartFTP.FavoritesManager; $rootFolder = $favMgr.RootFolder; $rootFolder.AddItem($fav); # Get the transfer queue $queue = $smartFTP.TransferQueue; # stop the queue if it isn't already if ($queue.State -ne 1) { $queue.Stop(); } # Stopped = 1 # clear the queue foreach($item in $queue.Items) { $queue.RemoveItem($item); } # set the thread count for the queue $queue.MaxWorkers = 20; #enable logging $queue.Log = "true"; $queue.LogFolder = $currFolder + "\\LOG"; # create new transfer item $newItem = $smartFTP.CreateObject("sfTransferQueue.TransferQueueItem"); # set the item as a folder and copy operation, $newItem.type = 2; #FOLDER = 2 $newItem.Operation = 1; #COPY = 1 # Set the source $newItem.Source.type = 1; #LOCAL = 1 $newItem.Source.Path = $source; # Set the destination $newItem.Destination.type = 2; #REMOTE = 2 $newItem.Destination.Path = $path; $newItem.Destination.FavoriteIdAsString = $fav.IdAsString; #links up to our connection favorite # and finally add it $queue.AddItemTail($newItem); Write-Host "STARTING" -foregroundcolor yellow -backgroundcolor black; $queue.Start(); while ($queue.Items.Count -ne 0) { Write-Host "Processing...bytes transfered: " $queue.TransferredBytes; Start-Sleep -s 2; #wait 2 seconds } # store the total bytes $totalBytes = $queue.TransferredBytes; # cleanup smartftp app $queue.Quit(); $smartFTP.Exit(); # parse logs # regex to find "[DATE/TIME] STOR FILENAME # which indicates a file transfer $regex = new-object System.Text.RegularExpressions.Regex("\[[\w\-\:]*?\]\sSTOR\s(.+?)\[",,[System.Text.RegularExpressions.RegexOptions]::SingleLine); $totalFiles = 0; Write-Host "Files Transfered" -foregroundcolor cyan -backgroundcolor black Get-ChildItem $queue.LogFolder -include *.log -Recurse | foreach ($_) { $currFile = Get-Content $_.fullname; $match = $regex.Matches($currFile); if ($match.Count -gt 0) { foreach($m in $match) { Write-Host $m.Groups[1]; } $totalFiles++; } remove-item $_.fullname -Force -Recurse ; } Write-Host "COMPLETED (total bytes: " $totalBytes ", total files: )" $totalFiles -foregroundcolor cyan -backgroundcolor black; "------------------------------------------------------" # cleanup COM Remove-Variable smartFTP

Examine RC3 Released

by Shannon Deminick 18. August 2010 18:30

Hopefully this will be a quick RC! I’m really hoping to release v1.0 RTM by early next week (latest). If you are able to help out with some testing it would be amazing!!

Here's what's new:

  • PDF Indexing
  • Easily implement custom data indexing outside of Umbraco
  • More XSLT Extensions for Umbraco
  • Some framework refactoring so a new DLL: Examine.LuceneEngine.dll which contains all of the Lucene.Net implementation
    • Because of this refactoring, if you've built your own providers, you may need to update our code to work, otherwise it is backwards compatible for most people.
  • More unit tests
  • More documentation

Get it while it’s hot! And don’t forget to read the release notes.

DOWNLOAD FROM CODEPLEX HERE

Categories: Examine | Umbraco