Full-stack web applications developer

Welcome to my blog

As I discover new things in my field and solve troublesome problems, I will attempt to document my finds here.

Full page screenshots with Python and Selenium

December 18, 2017, Arthur Pemberton0 Comments

Currently, none of the major Selenium drivers (browsers) support the ability to easily take a screenshot of an entire web page. The following function takes multiple screenshots through the viewport and scrolls between screenshots, then stitches the resulting images into a single PNG.

def save_fullpage_screenshot(driver, url, output_path, tmp_prefix='selenium_screenshot', tmp_suffix='.png'):
	Creates a full page screenshot using a selenium driver by scrolling and taking multiple screenshots,
	and stitching them into a single image.

	# get the page

	# get dimensions
	window_height = driver.execute_script('return window.innerHeight')
	scroll_height = driver.execute_script('return document.body.parentNode.scrollHeight')
	num = int( math.ceil( float(scroll_height) / float(window_height) ) )

	# get temp files
	tempfiles = []
	for i in xrange( num ):
		fd,path = tempfile.mkstemp(prefix='{0}-{1:02}-'.format(tmp_prefix, i+1), suffix=tmp_suffix)

		# take screenshots
		for i,path in enumerate(tempfiles):
			if i > 0:
				driver.execute_script( 'window.scrollBy(%d,%d)' % (0, window_height) )
		# stitch images together
		stiched = None
		for i,path in enumerate(tempfiles):
			img =
			w, h = img.size
			y = i * window_height
			if i == ( len(tempfiles) - 1 ):
				img = img.crop((0, h-(scroll_height % h), w, h))
				w, h = img.size
			if stiched is None:
				stiched ='RGB', (w, scroll_height))
			stiched.paste(img, (
				0, # x0
				y, # y0
				w, # x1
				y + h # y1
		# cleanup
		for path in tempfiles:
			if os.path.isfile(path):

	return output_path

The following libraries are required:

  • selenium
  • Pillow

Calling openssl genrsa from Python

September 28, 2017, Arthur Pemberton0 Comments

I have been writing some automation code a few dozen websites, and I wanted to generate SSL keys, and I couldn’t get the `subprocess` call to work properly, so I thought I would post the final solution. The argument to `communicate()` is critical.

import subprocess

filename = 'my.key'
numbits = 2048

cmd = ['openssl', 'genrsa', '-out', filename, str(numbits), '-passin', 'stdin']

p = subprocess.Popen(
	stdin = subprocess.PIPE,
	stdout = subprocess.PIPE,
	stderr = subprocess.PIPE,

out, err = p.communicate('\n')

Replicating Chilkat AES Cryptography with PyCrypto

November 22, 2016, Arthur Pemberton0 Comments

Today, I had the need to replicate an encrypted query string token to inter-operate with a third-party commercial application. I was able to determine the library, symmetrical algorithm and secret key being used to create the token. Turns out, it was an web application using the Chilkat .NET library to do the encryption and decryption. Specifically, it was the Chilkat AES (aka Rijndael) methods being used.
Read More

Installing pgAdmin4 Web on Centos 7

October 13, 2016, Arthur Pemberton0 Comments

Step 1: Install Packages

Install yum repo, you can the correct one at

yum install

Install the actual packages:

yum install pgadmin4-web

Step 2: Create system user

The packages do not create a user to run the web application now, so take some time to do so now:

useradd --create-home --home-dir /var/pgadmin --system --shell /sbin/nologin

Step 3: Run

It’s necessary to run to create the appropriate environement under the system user’s HOMEDIR:

su –shell /bin/sh -c “python /usr/lib/python2.7/site-packages/pgadmin4-web/” pgadmin

Step 4: Configure Apache HttpD

<VirtualHost *>

	CustomLog "logs/pgadmin-access_log" combined
	ErrorLog "logs/pgadmin-error_log"
	LogLevel error
	WSGIDaemonProcess pgadmin processes=1 threads=25 user=pgadmin group=pgadmin
	WSGIScriptAlias / /usr/lib/python2.7/site-packages/pgadmin4-web/pgAdmin4.wsgi

	<Directory /usr/lib/python2.7/site-packages/pgadmin4-web>
		WSGIProcessGroup pgadmin
		WSGIApplicationGroup %{GLOBAL}
		Require all granted

Openfire certificate import

September 27, 2016, Arthur Pemberton0 Comments

Working with SSL certificates in Openfire are surprisingly troublesome, at least as compared to popular web servers (Apache hTTPd, Nginx, IIS).

I recently needed to update an expiring certificate (having once, previously, figured out how to successfully import a purchased, SSL certificate.

This time I found a nice tutorial: Thanks to Christian for that.

I’ve made some slight modifications to his Shell script, and I just wanted to share

#! /bin/bash
## echo "stop openfire"
##/etc/init.d/openfire stop
echo "> deleting truststore and keystore"
test -e "${JavaDir}/truststore" && rm -f "${JavaDir}/truststore"
test -e "${JavaDir}/keystore" && rm -f "${JavaDir}/keystore"

echo "> merge domain certificate with CA certificate"
cat "${certdir}/${certname}.crt" "${certdir}/${certname}.ca-bundle" > "${certdir}/${certname}.combined.crt"

echo "> create a new trust store"
keytool -import -trustcacerts -storepass $PASS -alias "PositiveSSL" -file "${certdir}/${certname}.ca-bundle" -keystore "${JavaDir}/truststore"

echo "> create new p12 file"
openssl pkcs12 -export -in "${certdir}/${certname}.combined.crt" -inkey "${certdir}/${certname}.key" -out "${certdir}/${certname}.p12" -name "${certdomain}" -CAfile "${ca}" -passout pass:"${PASS}"

echo "> create new key store"
keytool -importkeystore -deststorepass "$PASS" -srcstorepass "$PASS" -destkeystore "${JavaDir}/keystore" -srckeystore "${certdir}/${certname}.p12" -srcstoretype PKCS12 -alias "${certdomain}"

echo "> change file perms"
chmod 644 "${JavaDir}/truststore" "${JavaDir}/keystore"

echo "> change ownership"
chown daemon:daemon "${JavaDir}/truststore" "${JavaDir}/keystore"

echo "> list directory"
ls -lha "${JavaDir}" *store*

## echo "start openfire"
##/etc/init.d/openfire start


A few quick notes:

  • This _will_ delete your existing truststore and keystore, back them up first
  • I disable stopping and starting of the OpenFire service, feel free to uncomment those
  • This version expects the CA’s chain certificate to be named in the format ``
  • This only creates an RSA certificate

Adding version numbers to your static files

June 12, 2016, Arthur Pemberton0 Comments

When I started with Django, the version at the time was 1.5. Back then, we prepended STATIC_URL to our static assets to reference them in our templates. With recent release, best practice is to you use the static [1]. I often find myself wanting to append a version number to my static files, at least my CSS and JS files to ensure that browser see my new versions when I push an update. In this post, I make use of the `static` tag and its URL building to easily append a version number.

The Code

The static tag uses the active storage engine to generate the URL for the static file, by default this is the StaticFilesStorage engine. So we’re going to want to subclass this engine and override the, url() method to generate our versioned URL.

# django imports
from django.conf import settings
from import StaticFilesStorage
from django.utils.encoding import filepath_to_uri
from django.utils.six.moves.urllib.parse import parse_qs
from django.utils.six.moves.urllib.parse import urlencode


# classes

class VersionedStaticFilesStorage(StaticFilesStorage):

	def url(self, name):
		Generates URL for static file.
		Requires STATICFILES_VERSION setting.
		# if there is a query string already, isolate it
			idx = name.index(u'?')
			qs = name[idx+1:]
			name = name[:idx]
		except ValueError:
			idx = -1
			qs = None
		# build a dictionary
		query = parse_qs(qs) if qs else {}
		# add in our version number
		# get url
		url = super(VersionedStaticFilesStorage, self).url(name)
		# rebuild query string
		qs = urlencode(query.items(), doseq=True)
		# return combined url
		return url + u'?' + qs


Add this code to a module, for example, in your app, and then reference it in the STATICFILES_STORAGE setting.


Be sure to create your version value in


And now you should have static URLs like this:

<script type="text/javascript" src="/static/js/bootstrap.min.js?v=2.0"></script>

Good luck, and happy coding.

Request specific URLs in Django

September 17, 2015, Arthur Pemberton0 Comments

Do you have a need to use different URL pattern sets based on specifics of an HTTP request? Recently, I needed to choose from a predefined URL pattern (ie. URLConf module) based on the domain name of the request, ie. HTTP_HOST. The URL patterns themselves are not dynamically set, just dynamically chosen.
Read More