Managing WebJobs with the Kudu API

I've posted before about how you can deploy a WebApp as a zip with the Kudu zip deploy API. It's a great way to deploy web apps and is one of the techniques I discuss for deploying miniblog.core.

But as well as allowing us to deploy our web apps, Kudu has an API for managing webjobs. With this API we can deploy and update new webjobs individually, as well as triggering them, configuring settings and even getting their execution history.

Three types of webjob

There are three main types of webjob that you can use. First there are triggered webjobs. These are webjobs that run on demand. Typically they will simply be a console app. You can trigger an execution of one of these webjobs with the Kudu webjobs API (we'll see an example later). A typical use case for this type of webjob might be some kind of support tool that you want to run on demand.

The second type is a scheduled webjob, which is actually just a triggered webjob with a schedule cron expression. The schedule is defined in a settings.json file that sits alongside your webjob executable. This type of webjob is great for periodic cleanup tasks that you need to run on a regular basis without explicitly needing to do anything to trigger them.

Finally there is a continuous webjob. This is an executable that will be run continuously - that is, it will be restarted for you if it exits. This is great for webjobs that are responding to queue messages. The webjob sits listening on one (or many) queues, and performs an action when a message appears on that queue. There's a helpful SDK that makes it easier to build this type of webjob, although I won't be discussing the use of that today.

Where are webjobs stored?

Creating a webjob simply involves dumping our webjob binaries into specially named folders. For a triggered (or scheduled) job, the folder is wwwroot\app_data\jobs\triggered\{job name}, and for a continuous job, it's wwwroot\app_data\jobs\continuous\{job name}. The webjobs host will look inside that folder and attempt to work out what executable it should run (based on a set of naming conventions).

Why the app_data folder? Well that's a special ASP.NET folder that is intended for storing your application data. The web server will not serve up the contents of this folder, so everything in there is safe. It's also considered a special case for deployments - since it might contain application generated data files, its contents won't get deleted or reset when you deploy a new version of your app.

An example scenario

Let's consider a very simple example where we have two web jobs that we want to host. One is a .NET core executable (Webjob1), the other is a regular .NET 4.6.2 framework console app (Webjob2). And we'll also deploy a ASP.NET Core Web API, just to show that you can host web jobs in the same "Azure Web App" instance as a regular web app, although you don't have to.

We'll use a combination of the Azure CLI and PowerShell for all the deployments, but these techniques can be used with anything that can make zip files and web requests.

Step 1 - Creating a web application

As always, with the Azure CLI, make sure you're logged in and have the right subscription selected first.

# log in to Azure CLI
az login
# make sure we are using the correct subscription
az account set -s "MySub"

And now let's create ourselves a resource group with an app service plan (free tier is fine here) and a webapp:

$resourceGroup = "WebJobsDemo"
$location = "North Europe"
$appName = "webjobsdemo"
$planName = "webjobsdemoplan"
$planSku = "F1" # allowed sku values B1, B2, B3, D1, F1, FREE, P1, P1V2, P2, P2V2, P3, P3V2, S1, S2, S3, SHARED.

# create resource group
az group create -n $resourceGroup -l $location

# create the app service plan
az appservice plan create -n $planName -g $resourceGroup -l $location --sku $planSku

# create the webapp
az webapp create -n $appName -g $resourceGroup --plan $planName

Step 2 - Get deployment credentials

We'll need the deployment credentials in order to call the Kudu web APIs. These can be easily retrieved with the Azure CLI making use of the query syntax which I discuss in my Azure CLI: Getting Started Pluralsight course

# get the credentials for deployment
$user = az webapp deployment list-publishing-profiles -n $appName -g $resourceGroup `
    --query "[?publishMethod=='MSDeploy'].userName" -o tsv

$pass = az webapp deployment list-publishing-profiles -n $appName -g $resourceGroup `
    --query "[?publishMethod=='MSDeploy'].userPWD" -o tsv

Step 3 - Build and zip the main web API

As I said, there is no requirement for our Azure "webapp" to actually contain a webapp. It could just host a bunch of webjobs. But to show that the two can co-exist, let's build and zip an ASP.NET Core web api application. I'm just using a very basic example app created with dotnet new webapi. We're using some .NET objects in PowerShell to perform the zip.

$publishFolder = "publish"

# publish the main API
dotnet publish MyWebApi -c Release -o $publishFolder

# make the zip for main API
$mainApiZip = "publish.zip"
if(Test-path $mainApiZip) {Remove-item $mainApiZip}
Add-Type -assembly "system.io.compression.filesystem"
[io.compression.zipfile]::CreateFromDirectory($publishFolder, $mainApiZip)

Step 4 - Deploy with Kudi zip deploy

The Azure CLI offers us a really nice and easy way to use the Kudu zip deploy API. We simly need to use the config-zip deployment source:

az webapp deployment source config-zip -n $appName -g $resourceGroup --src $mainApiZip

However, a regression in the Azure CLI 2.0.25 meant this was broken, so as an alternative you can just call the API directly with the following code, passing the credentials we retrieved earlier.

# set up deployment credentials
$creds = "$($user):$($pass)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($creds))
$basicAuthValue = "Basic $encodedCreds"

$Headers = @{
    Authorization = $basicAuthValue
}

# use kudu deploy from zip file
Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/zipdeploy -Headers $Headers `
    -InFile $mainApiZip -ContentType "multipart/form-data" -Method Post

If we want to verify that the deployment worked, we can get the URI of the web app, and call the values controller (which the default webapi template created for us):

# check its working
$apiUri = az webapp show -n $appName -g $resourceGroup --query "defaultHostName" -o tsv
Start-Process https://$apiUri/api/values

Step 5 - Build our first webjob

In our demo scenario we have two webjobs. The first (Webjob1) is a .NET Core command line app. The code is very simple, just echoing a message and including command line arguments

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Hello from Task 1 (.NET Core) with args [{0}]!", 
            string.Join('|',args));
    }
}

Since .NET Core apps are just DLLs, we need to help the webjobs host to know how to run it by creating a run.cmd batch file that calls the dotnet runtime and passes on any command line arguments. Note: You can get weird errors here if you have a UTF-8 encoded file. Make sure you save this batch file as ASCII.

@echo off
dotnet Webjob1.dll %*

Building and zipping this webjob is no different to what we did with the main web API:

# now lets build the .NET core webjob
dotnet publish Webjob1 -c Release

$task1zip = "task1.zip"
if(Test-path $task1zip) {Remove-item $task1zip}
[io.compression.zipfile]::CreateFromDirectory("Webjob1\bin\Release\netcoreapp2.0\publish\", $task1zip)

Step 6 - Deploy the webjob

Deploying a webjob using the Kudu Webjobs API is very similar to zip deploying the main webapp. We simpply need to provide one extra Content-Disposition header, and we use the PUT verb. We indicate that this is going to be a triggered web job, by including triggeredwebjobs in the path, and we also include the webjob name (in this case "Webjob1")

$ZipHeaders = @{
    Authorization = $basicAuthValue
    "Content-Disposition" = "attachment; filename=run.cmd"
}

# upload the job using the Kudu WebJobs API
Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob1 -Headers $ZipHeaders `
    -InFile $task1zip -ContentType "application/zip" -Method Put

To check it worked, you can visit the Kudu portal and explore the contents of the app_data folder or look at the web jobs page.

# launch Kudu portal
Start-Process https://$appName.scm.azurewebsites.net

We can also check by calling another web jobs API method to get all triggered jobs:

# get triggered jobs
Invoke-RestMethod -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs -Headers $Headers `
    -Method Get

Step 7 - Run the webjob

To run the webjob we can POST to the run endpoint for this triggered webjob. And we can optionally pass arguments in the query string. Don't forget to provide the content type or you'll get a 403 error.

# run the job
$resp = Invoke-WebRequest -Uri "https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob1/run?arguments=eggs bacon" -Headers $Headers `
    -Method Post -ContentType "multipart/form-data"

Assuming this worked, we'll get a 202 back, and it will include the URI of a job instance we can use to query the output of this job. From the output of that request we'll also get a URI we can call to request the log output, which we can use to see that our webjob successfully ran and got the arguments we passed it:

# output response includes a Location to get history:
if ($resp.RawContent -match "\nLocation\: (.+)\n")
{
    $historyLocation = $matches[1]
    $hist = Invoke-RestMethod -Uri $historyLocation -Headers $Headers -Method Get
    # $hist has status, start_time, end_time, duration, error_url etc
    # get the logs from output_url
    Invoke-RestMethod -Uri $hist.output_url -Headers $Headers -Method Get
}

We can also ask for all runs of this webjob with the /history endpoint:

# get history of all runs for this webjob
Invoke-RestMethod -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob1/history -Headers $Headers `
    -Method Get

Step 8 - Deploy and configure a scheduled webjob

For our second webjob, we're using a regular .NET console app running on the regular .NET framework. Here's the code

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Hello from task 2 (.NET Framework) with args [{0}]", 
            string.Join("|", args));
    }
}

We'll build it with MSBuild and create a zip, very similar to what we did with the first webjob:

# build the regular .net webjob
$msbuild = "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\msbuild.exe"
. $msbuild "Webjob2\Webjob2.csproj" /property:Configuration=Release

$task2zip = "task2.zip"
if(Test-path $task2zip) {Remove-item $task2zip}
[io.compression.zipfile]::CreateFromDirectory("Webjob2\bin\Release\", $task2zip)

And then upload it just like we did with the first webjob. Remember a "scheduled" webjob is just a special case of triggered webjob, so we use the triggeredwebjobs endpoint again:

# upload the web job
$ZipHeaders = @{
    Authorization = $basicAuthValue
    "Content-Disposition" = "attachment; filename=Webjob2.exe"
}

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob2 -Headers $ZipHeaders `
    -InFile $task2zip -ContentType "application/zip" -Method Put

Now if we'd included a settings.json in our zip file, with a cron expression, then this would already be a scheduled job with nothing further to do. But there's a very handy /settings endpoint that lets us push the contents of the settings file, which we can use to set the schedule. Here we'll set up our second webjob to run every five minutes.

$schedule = '{
  "schedule": "0 */5 * * * *"
}'

Invoke-RestMethod -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob2/settings -Headers $Headers `
    -Method Put -Body $schedule -ContentType "application/json"

The great thing about this approach is that we can change the schedule without having to push the whole webjob again. And even though this webjob is on a schedule, there's nothing to stop us running it on-demand as well if we want to.

Updating and deleting webjobs

It's very easy to update webjobs (or indeed the main API). You just zip up your new version of the webjob exactly as we did before and upload it through the API. The webjobs are left intact when a new version of the main web app is deployed, so it's safe to update that as well with the zip deploy API.

You can also easily delete webjobs if you no longer need them:

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/WebJob2 -Headers $Headers `
    -Method Delete

Summary

As you can see, the Kudu web jobs API makes it very straightforward, to deploy, run, query and update your webjobs. This makes it a convenient platform for running occasional maintenance tasks. We've seen in this post how this can be easily scripted in PowerShell with the Azure CLI, but you can of course use your preferred shell and language to call the same APIs.