Updates from October, 2012 Toggle Comment Threads | Keyboard Shortcuts

  • Richard 10:52 am on October 31, 2012 Permalink |  

    Adding Content to an Azure Package 

    When using CloudServices in Windows Azure, you frequently want to add extra files to your package, such as startup scripts you want to register, or additional content that’s required.

    Traditionally this has always been done by adding the file to the project, and setting the ‘Copy to Output Directory’ property to ‘Copy if Newer’. The file will now be included in the package.

    There is another way. You can right click on your role in your Cloud Project, and add content or folders directly. Items added here will automatically be included in the package, with no need to adjust the file properties.

    The 1.8 (October 2012) SDK release allows you to add a startup task here. Previous versions seemed to raise an error.

    The advantage of adding content here is it separates our the Azure concerns from your project. You can use a single codebase for deploying an application on-premises, or in the cloud, without cluttering your project with script files that aren’t needed on-prem. It also stops you from forgetting to set the ‘Copy to Output Directory’ property!

     
  • Richard 1:00 pm on October 19, 2012 Permalink |  

    Automating Cloud Service Deployment in Azure 

    The build and deployment of Cloud Services can be automated, so your continuous integration server can push updates automatically to your (test) infrastructure.

    First of all, build the solution, and use CSPack to package the deployment:

    %WINDIR%\Microsoft.NET\Framework\v4.0.30319\MSBuild SOLUTION.sln /p:Configuration=Release /t:Publish

    You will need to update the command with the name of your solution.

    Then using PowerShell, upload to staging, wait for the instances to start, swap staging and production slots, and delete the staging environment.

    $path = (Get-location).Path + "\Azure\bin\Release\app.publish"
    $hostedService = "YOUR_HOSTED_SERVICE"
    
    # deploy the package to staging slot
    New-AzureDeployment -ServiceName $hostedService -Package "$path\ServiceDefinition.cspkg" -Configuration "$path\ServiceConfiguration.Cloud.cscfg" -Slot Staging
    
    do {
    	$ready=0
    	$total=0
    
    	# query the status of the running instances
    	$list = (Get-AzureRole -ServiceName $hostedService -Slot Staging -InstanceDetails).InstanceStatus 
    
    	# count the number of ready instances
    	$list | foreach-object { IF ($_ -eq "ReadyRole") { $ready++ } }
    
    	# count the number in total
    	$list | foreach-object { $total++ } 
    
    	"$ready out of $total instances are ready"
    
    	# sleep for 10 seconds
    	Start-Sleep -s 10
    }
    while ($ready -ne $total)
    
    # swap staging and production
    Move-AzureDeployment -ServiceName $hostedService
    
    # remove the staging slot
    Remove-AzureDeployment -ServiceName $hostedService -Slot Staging -Force

    Simple!

     
  • Richard 12:40 pm on October 12, 2012 Permalink |  

    Pub/Sub with Redis on Azure 

    After getting publish/subscribe working with ZeroMQ, I turned to Redis (mainly for a working Node.JS client). Redis is an in-memory key-value store, and includes a pub/sub feature.

    Redis is easy to get working on Azure, however, I felt the need to make it even easier with a plugin.

    Installing Redis on Azure

    To create a new server with Redis installed, create a Cloud Project with a Worker Role, use APM to add the Redis Plugin.

    > apm install Redis

    Then add Redis as a plugin in your Service Definition file:

    <WorkerRole name="WorkerRole1" vmsize="Small">
      <Imports>
        <Import moduleName="Diagnostics" />
        <Import moduleName="Redis" />
      </Imports>
    </WorkerRole>

    Done!

    Setting up the Subscriber (in Node.js)

    Install the Redis module using NPM:

    > npm install redis

    In your node code, you can subscribe to messages like this:

    var redis = require("redis");
    receiveclient = redis.createClient(6379, "WorkerRole1_IN_0");
    receiveclient.on("message", function (channel, message) {
      // do something with the message
    });
    receiveclient.subscribe("pub");

    And you can publish messages like this (using a separate client).

    sendclient = redis.createClient(6379, "WorkerRole1_IN_0");
    sendclient.publish("pub", message);

    Discovery

    If you’ve got Redis running on the same machine as the Node process, there is no need to specify the port and server name when creating a client. However, if this isn’t the case, Node needs to know the ip address of the instance hosting Redis. If your process is hosted by IISNode, then you can call the Azure API to discover the topology of your deployment, and find the correct IP address. This isn’t the case for me. I’m using  raw sockets and I can’t use IISNode.

    So the question is, how do you discover the IP address of your Redis instance?

    I created another plugin (HostsWriter) which continually updates the hosts file in Windows, with the IP address of each instance in the deployment.

    This means you can just refer to instances by their instance name, and Windows will connect you to the right IP address.

    To install, run this command with APM :

    > apm install HostsWriter

    And add it to the Service Definition file of the WorkerRole hosting your Node.js script.

    <WorkerRole name="WorkerRole2" vmsize="Small">
      <Imports>
        <Import moduleName="Diagnostics" />
        <Import moduleName="HostsWriter" />
      </Imports>
    </WorkerRole>

    Conclusion

    Redis is really easy to set up and use. It also works with Node, and module is 100% JavaScript, which is a big tick for me. Whether it’s as fast as ZeroMQ, I’m not sure, but it’s my current favourite!

     
  • Richard 12:38 pm on October 10, 2012 Permalink |  

    Pub/Sub with ZeroMQ on Azure 

    I wanted to do real-time publish/subscribe, and although the Azure Service Bus supports this with topics and subscriptions, it seems to be geared towards guaranteed delivery rather than real-time. You need to make a request to receive a message, which didn’t quite fit the real-time nature of my application. Enter ZeroMQ.

    ZeroMQ (or ØMQ) is a library for working with sockets. There are clients for a variety of languages, including C# – although I can’t get the node module to build :¬(. It’s a fairly cool piece of technology, and takes all the pain out of socket programming.

    ZeroMQ needs to be installed on the machine, and some C++ runtime DLLs are also required. To make this super easy I created a plugin in the AzurePluginLibrary to do just this. To install, just run this command using APM.

    > apm install ZeroMQ

    Setting up the Publisher

    Next I created a Cloud Project, and a WorkerRole to act as my publisher. To install the ZeroMQ references, you just need to run this nuget command in Visual Studio:

    PM> Install-Package clrzmq -Pre

    In the ServiceDefinition file for the Worker, add the ZeroMQ plugin, and two endpoints (the port numbers don’t matter):

    <WorkerRole name="WorkerRole1" vmsize="Small">
      <Imports>
        <Import moduleName="Diagnostics" />
        <Import moduleName="ZeroMQ" />
      </Imports>
      <Endpoints>
        <InternalEndpoint name="Endpoint1" protocol="tcp" />
        <InternalEndpoint name="Endpoint2" protocol="tcp" />
      </Endpoints>
    </WorkerRole>

    Now you just need to write the C# code to receive a message from a client, and publish to all connected clients. I put this code in the ‘Run’ method of my RoleEntryPoint.

    public override void Run()
    {
      var context = ZmqContext.Create();
    
      // set up the socket which publishes messages to subscribers
      var endpoint1 = GetEndpoint("Endpoint1");
      var publisher = context.CreateSocket(SocketType.PUB);
      publisher.Bind(string.Format("tcp://*:{0}", endpoint1.IPEndpoint.Port));
    
      // set up the socket which receives messages from subscribers
      var endpoint2 = GetEndpoint("Endpoint2");
      var receiver = context.CreateSocket(SocketType.PULL);
      receiver.Bind(string.Format("tcp://*:{0}", endpoint2.IPEndpoint.Port));
    
      // publish every message you receive!
      while (true)
      {
        var message = receiver.Receive(Encoding.UTF8);
        publisher.Send(message, Encoding.UTF8);
      }
    }

    The ‘GetEndpoint’ function in this case just returns the required RoleInstanceEndpoint of the current instance.

    private RoleInstanceEndpoint GetEndpoint(string name)
    {
        return RoleEnvironment
            .CurrentRoleInstance
            .InstanceEndpoints
            .Where(x => x.Key == name)
            .Select(x => x.Value).FirstOrDefault();
    }

    Setting up the Subscriber

    I then created a WebRole which acts as the subscriber. This also needs the ServiceDefinition file updating to include the ZeroMQ plugin. I added the code into the Global.asax, but for simplicity I have extracted it out into a single code block for this example.

    var context = ZmqContext.Create();
    
    // set up the socket which subscribed to messages from the publisher
    var endpoint1 = GetEndpoint("Endpoint1");
    var subscriber = this.Context.CreateSocket(SocketType.SUB);
    subscriber.Connect(string.Format("tcp://{0}:{1}", endpoint1.IPEndpoint.Address, endpoint1.IPEndpoint.Port));
    subscriber.Identity = Guid.NewGuid().ToByteArray();
    subscriber.SubscribeAll();
    
    // set up the socket which sends messages to the publisher
    var endpoint2 = GetEndpoint("Endpoint2");
    var sender = this.Context.CreateSocket(SocketType.PUSH);
    sender.Connect(string.Format("tcp://{0}:{1}", endpoint2.IPEndpoint.Address, endpoint2.IPEndpoint.Port));
    
    while (true)
    {
        var message = subsriber.Receive(Encoding.UTF8);
        // do something with the messages you receive
    }

    To send a message, you just call:

    sender.Send(message, Encoding.UTF8);

    In this code snipper, the GetEndpoint function is slightly different:

    private RoleInstanceEndpoint GetEndpoint(string name)
    {
        return RoleEnvironment
            .Roles["WorkerRole1"]
            .Instances.First()
            .InstanceEndpoints
            .Where(x => x.Key == name)
            .Select(x => x.Value)
            .FirstOrDefault();
    }

    Conclusion

    This sample shows a simple pub/sub scenario with ZeroMQ in Azure. Whenever the ‘Send’ function is called, the message will almost instantly pop out at each of the subscribers. Network disruption and reconnecting the socket is all handled for you. ZeroMQ is a powerful tool. This is just one way of configuring it, there are a number of different scenarios that it can support.

     
    • Pedro 5:58 pm on October 19, 2012 Permalink | Log in to Reply

      Hi, nice article!

      How would you proceed to setup a webservice that publishes to the worker role, waits for the answer (a)synchronusly and returns it back to the client?
      I’m guessing both web and worker roles would have to be publishers and subscribers?
      I’m trying to replace Azure’s ServiceBus funcionality with ZMQ.

      Thanks!

    • Richard 8:03 pm on October 21, 2012 Permalink | Log in to Reply

      Hi, you’re probably best off with the REQ/REP socket modes. It’s the most simple ZeroMQ configuration. This chapter has an example which might help: http://zguide2.zeromq.org/page:all#Ask-and-Ye-Shall-Receive

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel