Azure Simplified Part 4: Using Azure Queue Storage
May 20, 2010 6 Comments
We’ve seen Azure table and blob storage, let’s look at queue storage today. The first four steps are the same as before,
1. Open up VisualStudio 2008 (run as Administrator) and create a new ‘Cloud Service’ project,
2. Create an ASP.NET WebRole,
3. Add a new data connection string. To do this, go to CloudService1 –> Roles –> WebRole1 –> right click Properties –> Settings –> Add Setting. Type in ‘DataConnectionString’ and change the Type column to ‘Connection String’. Then click on the ‘…’ at the far right and choose ‘Use development storage’.
4. Nasty hack alert! Add the following code to the WebRole.cs’s OnStart method,
public override bool OnStart() { DiagnosticMonitor.Start("DiagnosticsConnectionString"); // For information on handling configuration changes // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357. RoleEnvironment.Changing += RoleEnvironmentChanging; // This code sets up a handler to update CloudStorageAccount instances when their corresponding // configuration settings change in the service configuration file. CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) => { // Provide the configSetter with the initial value configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)); RoleEnvironment.Changed += (sender, arg) => { if (arg.Changes.OfType<RoleEnvironmentConfigurationSettingChange>() .Any((change) => (change.ConfigurationSettingName == configName))) { // The corresponding configuration setting has changed, propagate the value if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName))) { RoleEnvironment.RequestRecycle(); } } }; }); return base.OnStart(); }
5. Now add the following HTML to the Default.aspx file,
<body> <form id="form1" runat="server"> <div> <asp:Label id="Label1" Text="Insert this text message into queue:" runat="server" /> <asp:TextBox id="TextBox1" runat="server" /> <asp:Button ID="Button1" Text="Insert message" runat="server" /> <br /> <br /> <asp:Label id="Label2" Text="Insert file as binary data:" runat="server" /> <asp:FileUpload ID="fileUploadControl" runat="server" /> <asp:Button ID="Button2" Text="Insert binary data" runat="server" /> <br /> <br /> <asp:Button ID="Button3" Text="Retrieve message" runat="server" /> <asp:Label id="Label3" runat="server" /> <br /> <br /> <asp:Repeater id="Repeater1" runat="server"> <HeaderTemplate> <table border="1"> <tr><td><b>Queue Messages</b></td></tr> </HeaderTemplate> <ItemTemplate> <tr> <td> <asp:Label runat="server" Text="<%# Container.DataItem %>" /> </td> </tr> </ItemTemplate> <FooterTemplate> </table> </FooterTemplate> </asp:Repeater> </div> </form> </body>
6. Add the following code to the Default.aspx.cs file,
public partial class _Default : System.Web.UI.Page { // Manage account information CloudStorageAccount storageAccount = null; // Cloud queue CloudQueue cloudQueue = null; // Cloud client CloudQueueClient queueClient = null; protected void Page_Load(object sender, EventArgs e) { // Init the contexts this.storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString"); this.queueClient = storageAccount.CreateCloudQueueClient(); // Get and create the container this.cloudQueue = this.queueClient.GetQueueReference("myqueue"); this.cloudQueue.CreateIfNotExist(); // Hook up the buttons this.Button1.Click += new EventHandler(InsertTextMessage); this.Button2.Click += new EventHandler(InsertByteArrayMessage); this.Button3.Click += new EventHandler(RetrieveMessage); this.DisplayMesssages(); } // Simulate a consumer of messages: Retrieve a message from the queue and process it private void RetrieveMessage(object sender, EventArgs e) { // Get a single message from the queue // We can also get multiple messages using GetMessages CloudQueueMessage msg = this.cloudQueue.GetMessage(); this.Label3.Text = "Retrieved message: " + msg.AsString; // Done processing the message, delete the message from the queue this.cloudQueue.DeleteMessage(msg); this.DisplayMesssages(); } // Simulate a producer of messages: Insert text message into queue private void InsertTextMessage(object sender, EventArgs e) { CloudQueueMessage msg = new CloudQueueMessage(this.TextBox1.Text); this.cloudQueue.AddMessage(msg); this.DisplayMesssages(); } // Simulate a producer of messages: Insert binary message into queue private void InsertByteArrayMessage(object sender, EventArgs e) { // The file size needs to be less than 8KB // NOTE: Place the file in the following folder, else file read will fail, // [ProjectPath]\CloudService1\CloudService1\bin\Debug\CloudService1.csx\roles\WebRole1 CloudQueueMessage msg = new CloudQueueMessage(File.ReadAllBytes(fileUploadControl.FileName)); this.cloudQueue.AddMessage(msg); this.DisplayMesssages(); } // Peek into the queue for messages private void DisplayMesssages() { // Get the first 5 messages in the queue, but don’t remove them var msgs = this.cloudQueue.PeekMessages(5); var cloudList = new List<string>(); foreach (var msg in msgs) { cloudList.Add("Message ID: " + msg.Id + "; Message: " + msg.AsString + "; Message insertion time: " + msg.InsertionTime); } // Bind the repeater this.Repeater1.DataSource = cloudList; this.Repeater1.DataBind(); } }
7. Running the app you should get,
The Azure queue is a FIFO queue, so when you peek/get a message, it returns the first message (earliest message by time) that was inserted into the queue, problem is this is not guaranteed, so you need to carefully plan your application.
There is also the idea of message visibility, when a consumer pulls a message from the queue using GetMessage(), a few things happen,
1. The consumer notifies the queue that it wants a time of (the default) 30 seconds to process the message (and finally delete it from the queue). This can be changed by the consumer using the GetMessage(TimeSpan visibilityTimout) overload.
2. The queue gives the consumer a unique ‘popreceipt’ string (to track the most recent requester of this particular message) for the GetMessage operation, stores this ‘popreceipt’ id, and marks the message as invisible to other consumers for the duration of the what the consumer wants.
So when a new consumer requests a message, the next available visible message is sent to it. In case a consumer asks for a 30sec processing time, but fails to delete the message during this time, the message is marked visible again by the queue and is made available to the next consumer. If the consumer tries to delete a message after the timeout, the queue checks the ‘popreceipt’ to see if it indeed is the most recent requester of the message, if not an exception is thrown.
About soumya chattopadhyay
Pingback: Azure Simplified Series « I.Net
Pingback: Azure Simplified Part 5: Using REST API for Azure Queue Storage « I.Net
Very good Post.
Thanks you for posting.
Was just wondering if you do GetMessage() and there is no message in the queue present. Will it throw error or just return null.
Will figure out anyway.
Thanks again for your post.
it will return a null
thank u for the post
very informative
I am a student working on a project to parallelize information processing and this post really really helped. Thanks a lot.