‚Äč claudinebroke.it

Claudine Broke It

Dependency version reporting with Python, Github (or Bitbucket) & Slack

Posted on 25 October 2016

Monitoring Chicken

Get the code on GitHub

I like sharing the work that I do through packaged modules. If I do a great job developing and maintaining my bit, then other devs can concentrate their time and energy making their own domain better. Not only that, but sharing codebase also opens great communication channels across teams and permits deployment of new features and bugfixes quickly and easily across a network.

However, maintaining modules can sometimes be a daunting task; most modules are too small to deserve a dedicated team, which means that the developer b ehind the codebase often becomes the product owner, the support team, the development team, and so on.

So as the dev behind a little Composer package that is now a major dependency for many in-house products, I get asked this question quite a lot:

Does X product support the Y feature yet?

To which I usually reply:

That feature was introduced as part of release version X.X, depends if the product followed up on the upgrade. Let’s go check!

When multiple products integrate your package, chances are you won’t know by heart what specific tag release each has deployed on production. Which means that, to answer the above question, you will either:

[1] Go on your company's Git repository and hope you can find -- and have access to -- the repository for the project. You then attempt, through commit history and pure Sherlock Holmes deducing skills, to determine the current branch deployed in production. Once that’s done, find the composer.lock file and ctrl+f your package's name to retrieve the version number.

[2] You contact someone on the product's team (if you know anyone on the team, if not, contact someone who might know someone on the team). You move on to another task. And another. You live a wonderful, fulfilling life. You retire with your sweetheart, get that cute dog you've always wanted, travel all over the country and meet wonderful individuals. At the top of this challenging mountain you just climbed you get a message back from that guy from that team: "Hey, sorry was busy. I dunno what version we're currently using. Let me ask around."

So after quite a bit of [1] and [2], I’ve decided to attempt to automate this process a bit and make a small tool that, on a daily basis, prints to my team’s Slack channel a report containing the version of the package integrated as a dependency repositories I provide as a configuration.

Outline

You’ll see, getting this done is incredibly simple. Since Github and Stash offer RAW file view, you can just execute a cURL request to a URL to get the json content of a dependency file. Here’s an example using a personal project of mine:

https://raw.githubusercontent.com/apricat/qux/master/package.json


{
  "name": "qux",
  "version": "1.0.0",
  "description": "Where you go to study",
  "main": "server.js",
  "author": "Claudine Lamothe",
  "license": "ISC",
  "dependencies": {
    "angular2": "^2.0.0-beta.17",
    "body-parser": "^1.15.2",
    "bootstrap": "^4.0.0-alpha.4",
    "es6-shim": "^0.35.1",
    "express": "^4.14.0",
    "method-override": "^2.3.6",
    "mongoose": "^4.6.2",
    "morgan": "^1.7.0",
    "reflect-metadata": "^0.1.2",
    "rxjs": "^5.0.0-beta.6",
    "zone.js": "^0.6.25"
  }
}

As you can see, this isn’t a Composer file, it’s NPM. This just goes to show that you can customise the script on any type of dependency manager as long as it uses json format.

So now, all we need is to parse the file for a given object key and append the result to a Slack message. To do so, I used the SlackClient python library because it is incredibly easy to use. For the full documentation: https://github.com/slackhq/python-slackclient

Dependencies

Let’s start by installing our dependencies. In your command line console, type the following two commands:


pip install requests
pip install slackclient

Let’s create a file and name it configs.py. This is where we will also set the url to the RAW dependency files we wish to parse, and the name of the packages we wish to include in our report. Note that the index key for the repositories object is simply used as a title for the report’s section.


slack = {
	'token' : ,
	'channel' : <#yourChannel or @yourUsername>,
	'username' : 'version-monitoring-chikin'
}
repositories = {
	'Qux': 'https://raw.githubusercontent.com/apricat/qux/master/package.json'
}
packages = ['mongoose']

Now let’s create a new file and name it core.py.


import requests
from slackclient import SlackClient
import configs

sc = SlackClient(configs.slack['token'])

sc.api_call(
	"chat.postMessage",
	channel=configs.slack['channel'],
	text='Good morning :sparkles: \nHere is your daily _Package Version_ report:',
	username=configs.slack['username'],
	icon_emoji=':rooster:'
)

This will send a nice little message to our channel with a chicken emoji (crucial). Test it out! Run the following command in your project's directory:


python core.py

After the SlackClient’s api call, loop through the list of repositories we are monitoring, and use the requests module to GET the json file’s content:


for repositoryName, repositoryPath in configs.repositories.items():
  r = requests.get(repositoryPath)

Note that if you are using BitBucket, you can add you auth header like so in the request:


r = requests.get(v, auth=(, ))

Let’s start building our reply message:


message += '*' + repositoryName + '*\n';

And of course, let’s not forget validation:


if r.status_code != 200 :
  message += "_Error " + str(r.status_code) + " - Failed retrieving repository._\n"
	continue

try:
  data = r.json()
except:
  message += "_Error " + str(r.status_code) + " - Failed retrieving JSON data from repository._\n"
  continue

The try: except: statement is where I fell in love with Python:

Easier to ask forgiveness than permission.

What this means is that you don’t need to validate against multiple conditions before executing a code. If it fails for any reason, output an error message in the report.

Finally, we can parse the json response for the dependencies we want to monitor:


for package in configs.packages:
  message += '`' + package + '` : '

  try:
    message += '`' + data['dependencies'][package] + '`\n'
  except KeyError:
    message += "_Unable to find requirement in composer.json file._\n"

Note that here I’m checking for ‘dependencies’ which is because we’re checking npm dependencies as opposed to Composer dependencies. For Composer, change the index value to ‘require’.

Lastly, send the message to Slack:


sc.api_call(
    'chat.postMessage',
    channel=configs.slack['channel'],
    text=message,
    username=configs.slack['username'],
    icon_emoji=':rooster:'
)

In your command line, navigate to the project’s directory and enter the following:


python core.py

Slack message received!

Version-monitoring-chikin

Voila! Version-monitoring-chikin found that on the Qux project, Mongoose is set to ^4.6.2. You can then adapt the code in a multitude of ways like, for example, to look for the current tag release of your package and compare it.

Here’s the full code on my GitHub repository:

https://github.com/apricat/version-monitoring-chicken/releases/tag/v1.0

Thanks for reading!