I have created several vRealize Operations (vROps) alerts in the past, mainly using the Log File Plugin and Standard Email Plugin. However, I recently had someone ask for more information on using the Rest Notification Plugin. I hadn’t used this, so I started looking for more detail on how to get started.
I found a couple really good blog posts on this, specifically https://blogs.vmware.com/management/2017/01/vrealize-webhooks-infinite-integrations.html and https://blogs.vmware.com/management/2018/02/extending-vrealize-operations-collaboration-tools-restful-apis-webhook-shims-david-davis-vrealize-operations-post-50.html . Both of these posts describe using an intermediary to accept what vROps is sending and convert it into a format that another endpoint expects. There are a handful of integrations provided, so I started looking at one that I could test with. The following post will describe the steps to get this working.
The Test Service
For testing, I’m going to use vROps to send an alert to Slack. This was pretty straight forward, I created a new channel where I wanted the alerts to appear, and then created a new incoming webhook by visiting https://api.slack.com/apps/new. I created a new app called vrealize-bot using my workspace. Once the app was created, I toggled on the ‘incoming webhooks’ feature, and mapped to my channel. This resulted in a webhook URL that looked like this:
https://hooks.slack.com/services/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
To confirm this was working, I used a quick PowerShell script to try and post to that webhook URL. This isn’t needed, but did prove that my web hook was correctly created.
Invoke-WebRequest `
-Uri $webhookURL `
-Method POST `
-Headers @{"Content-type"="application/json"} `
-Body (@{"text"="This is a test"}|ConvertTo-Json)
The ‘Shim’
We need a piece of code to covert from the vROps Rest output into a format Slack will accept. The first blog post mentioned above calls out a prebuilt tool to do this — called the loginsightwebhookdemo. There are instructions available on getting this running, but the easiest route for me was to use the docker image. I started by downloading the Photon OS 3.0 OVA, deployed it to an ESXi host, and then enabled docker using these instructions. I ran three commands… only the middle one is required, the other two will just show some supplemental info.
systemctl status docker systemctl start docker docker in
Once docker is running, you can start the webhook-shims container using these instructions. As described in the instructions, you launch the bash shell, which gives you the ability to edit files in the container file system to add things like our Slack API URL. If you choose this route, once the files are edited in the loginsightwebhookdemo directory, you’ll need to run ./runserver.py from the webhook-shims directory. However, since we are only using the Slack shim in this example, there is an easier way. All we need to do is pull & run the container using these commands:
docker pull vmware/webhook-shims docker run -it -p 5001:5001 vmware/webhook-shims
With the container running we can access its info page at http://dockerhostnameorip:5001/. This will show everything is up and running and that you can connect to the website.
The vROps Alerts
From the Alerts > Alert Settings > Notification Settings area, we can add a new rule. We will select the Rest Notification Plugin method and add a new instance. We’ll name our instance SlackWebhook_vrealize-bot. We could enter anything we want here, but want to be descriptive as possible. This shows the service we are using, how we are connecting, and the application that will be doing the posting, which seems sufficient. The URL is where the magic happens. We’ll enter a URL like this:
http://dockerhostnameorip:5001/endpoint/slack/TTTTTTTTT/BBBBBBBBB/alphaNumericStr0ngOfText
This is the name of the host our container is on, the port that the service is exposed on, the endpoint/slack is to specifies which shim we want to use, and the TTTTTTTTT/BBBBBBBBB/alphaNumericStr0ngOfText comes from our Slack webhook at the beginning of the article. We will leave the username and password blank (all that authentication is done in our custom webhook URL). For content type we’ll select application/json. Pressing TEST should result in two new posts to our Slack channel.
Now all we need to finish up is to define the filtering criteria for which alerts we want to be sent to Slack. For testing, I just set criticality to Immediate or Critical, but we will likely want to narrow that down over time as it is a bit too chatty.
