Catches when Expecting Exceptions in Django Unit Tests

To cover all bases when writing a suite of unit tests, you need to test for the exceptional cases. However, handling exceptions can break the usual flow of the test case and confuse Django.

Example scenario: unique_together

For example, we have an ecommerce site with many products serving multiple countries, which may have different national languages. Our products may have a description written in different languages, but only one description per (product, language) pair.

We can set up a unique_together constraint to enforce that unique pairing:

class Description(models.Model):
    product = models.ForeignKey("Product")
    language = models.ForeignKey("countries.Language")

    class Meta:
        unique_together = ("product", "language")

    subtitle = models.CharField(...)
    body = models.CharField(...)
    ...

Developer chooses AssertRaises()

If the unique_together rule is violated, Django will raise an IntegrityError. A unit test can verify that this occurs using assertRaises() on a lambda function:

def test_unique_product_description(self):
   desc1 = DescriptionFactory(self.prod1, self.lang1)
   self.assertRaises(IntegrityError, lambda:
      desc2 = DescriptionFactory(self.prod1, self.lang1)

The assertion passes, but the test will fail with a new exception.

A wild TransactionManagementError appears!

Raising the exception when creating a new object will break the current database transaction, causing further queries to be invalid. The next code that accesses the DB - probably the test teardown - will cause a TransactionManagementError to be thrown:

Traceback (most recent call last):
File ".../test_....py", line 29, in tearDown
   ...
File ...
   ...
File ".../django/db/backends/__init.py, line 386, in validate_no_broken_transaction
An error occurred in the current transaction.
TransactionManagementError: An error occurred in the current transaction.
You can't execute queries until the end of the 'atomic' block.

Developer used transaction.atomic. It's super effective!

Wrapping the test (or just the assertion) in its own transaction will prevent the TransactionManagementError from occurring, as only the inner transaction will be affected by the IntegrityError:

def test_unique_product_description(self):
   desc1 = DescriptionFactory(self.prod1, self.lang1)
   with transaction.atomic():
       self.assertRaises(IntegrityError, lambda:
          desc2 = DescriptionFactory(self.prod1, self.lang1)

You don't have to catch 'em all: Another solution

Another way to fix this issue is to subclass your test from TransactionTestCase instead of the usual TestCase. Despite the name, TransactionTestCase doesn't use DB transactions to reset between tests; instead it truncates the tables. This may make the test slower for some cases, but will be more convenient if you are dealing with many IntegrityErrors in the one test. See the Django Documentation for more details on the difference between the two classes.

Webpack Your Things

We very recently finished migrating our front-end build process to Webpack. As with any reasonably sized codebase, it's always a little more complex than the 3 line examples in how-to guides. This post will list some of the higher-level things I learned during this undertaking, the next one in the Webpack Series will detail some specific quirks and solutions.

Resources

I was largely able to do this by leveraging the hard work of some clever heroes. This very conveniently timed blog post covered a lot of what was needed. JLongster also has some very good tips, helpful for more than just backend apps. Regarding documentation, the widely-cited Webpack How-to gives a pretty concise overview of most things you will need. And of course, the docs have a lot of information. Sometimes too much. But usually most things you need are listed there. Occasionally something isn't, which brings me to lesson one.

Lesson 1: cd node_modules

One of the biggest things I learned in this undertaking isn't limited just to Webpack, and helped fix a few other things. Previously I had treated the node_modules folder as a black box - just npm install and be on my way. This is fine for everyday usage, but when you hit barriers or bugs sometimes you need to do some digging. Rather than throwing random inputs at a black box to measure the effect, you can just crack it open.

A good example of this is the CommonsChunkPlugin, which is documented thusly:

If omitted and options.async or options.children is set all chunks are used, elsewise options.filename is used as chunk name
— chunk.name definition

I found this sentence somewhat confusing, but easy to clarify by reading the code that checks this. If/else and some variable assignments and straightforward things to follow. And the nature of Webpack modules means they are generally quite small, if all else fails just console.log everything.

Note this doesn't necessarily mean you must always open the box and understand the internal implementation of libs you are using. But it is reassuring to know that you can.

Lesson 2: Use Tables

For visualising data, never for layout. The quite excellent webpack analyse tool provides a tonne of useful data to improve your module situation. The crazy tree-view animations look awesome, and are animated and zoomable. But they can quickly spiral out of control into a meaningless ball of branches. There are fortunately table views for all these pages as well. While repeatedly processing a bunch of lists to try minimising file sizes isn't the most romantic task, you can sort and group a lot more easily. Conversely, the Chunks tree view stays parseable for a longer time (as you will have fewer chunks than modules). It can give a quick overview if any of your chunks are ballooning out of control, and the accompanying table used for more automated analysis.

Trees: Awesome, albeit unclear

Lesson 3: Measuring Victory

As with any code change, the best measure of success is not breaking anything. In this case our ideal outcome was the change was completely invisible to end users, assuming everything deploys and the site still works. Beyond that it was meant to simplify frontend development for all of our devs, again a success. We have simple build & watch tasks without global Node package requirements (except npm). We were able to deploy our first React component by adding a single line to process JSX. So we don't (yet) have a quantifiable metric for the success of this adventure into the world of Webpack. But as someone who formerly complained about writing build tasks, it has been fun.


In the next post I will drill down into a few specific issues encountered and how they were fixed, and some other useful features and tricks. If you have any advice or thoughts, we'd love to work with you - careers.

Top 3 lessons from CssConfAu

We recently got the opportunity to attend cssconf in Melbourne, followed by Decompress (a day of lightning talks and hacking).

I found it to be a really rewarding experience. The material presented was really relevant to my work and I got to pick the brains of some of the speakers during break time :)

Here's a list of my top 3 takeaways from the CSS Conf AU by category:

  1. CSS & Usability & Accessibility
    • 4 1/2 of theming css - Depending on your needs.
    • Everyone is responsible for "UX" - that includes devs!
    • Every hour you spend making the web faster, more accessible and easier to use, turns into days of time saved by real people
    • SVGs are what we should eat for breakfast every morning
    • Pick your colors wisely, because they might increase/decrease your accessibility
  2. Web Page performance:
    • Cram your initial view into the first 14kb
    • Eliminate any non-critical resources blocking your critical path
    • Be responsible with your responsive design
    • Perceived performance > Actual performance
  3. Animations:
    • Improves your user experience
    • Animate exclusively on opacity and transforms
    • Perceived speed > Actual speed
    • Animations help infer context from revealed information
    • Use animations instead of gifs!

After following all the speakers on Twitter, I happened upon a couple of good resources unbeknownst to me, and I’ve added them to my Easter break reading/review/code list of resources:

  1. Front End Guidelines by Benjamin de Cock
    https://github.com/bendc/frontend-guidelines

  2. CSS Guidelines by Harry Roberts
    http://cssguidelin.es/

  3. Stray articles I’ve missed from Filament Group
    http://www.filamentgroup.com/lab/
  4. A guide to SVG Animations by Sara Soueidan (+ her other articles)
    https://css-tricks.com/guide-svg-animations-smil/

Like what we do? Join us then.

ReactJS + Flux Meetup

Yesterday we had the pleasure of hosting the first ReactJS Melbourne meetup and we're absolutely thrilled with the turn out!

For those who missed out, here are the slides from the Kogan.com prototype demo we presented yesterday:

We'd like to thank everyone who came, and we look forward to seeing what you create with these tools!

Here are some snaps from the event:


Like the sound of how we work? Check out our Careers Page!

A Dashing Radiator

Our information radiator or dashboard is a powerful tool for our technology team. It is used to give a quick visual representation of the current state of things, and an anchor point for the team’s most important statistics and state. We are never short of lots of data, so without this dashboard we would need to create a habit of seeking out information from various sources, like New Relic, Sentry or Google Analytics.

Our radiator also creates conversations with people outside the team about the metrics that matter the most to us.

Tool

Here at Kogan.com, we are using the framework "Dashing" by the people at Shopify to power our dashboard, and we have it deployed on Heroku.

Dashing was our tool of choice as it has a decent amount of community contributed widgets to pull data from all different sources, like Google Analytics and New Relic.

Red/Green

To make our radiator easy to read and to draw our attention when it’s needed, we opted for a red/green visual system. If a widget is green it is good, if it turns red this indicates to us something that needs attention. This also means even if you are too far away to read the details of the widgets, you can still tell the overall state.

The standard widgets don’t do this out of the box, so we modified them. Here are some of our modified widgets:

Meter Widget

You can grab the code here or install it with:

$ dashing install e81a30b2ededb434062c
Time Since Last Widget

At Kogan, we like to deploy often, and this widget lets us know how long its been since we last deployed. You can grab the code here or install it with:

$ dashing install 656f8dc218d3e11d238a

Security

A concern we had with Dashing jobs was that it encourages people to hard-code the credentials of your third party sources into the ruby code, which is not good if you want to put the code in source control for others to see. To combat this, we put the credentials in environment variables and changed the code to use Env['VARIABLE_NAME'] notation to get them in the code.

This allows you to distribute a sourceable .sh script to those who need it, while keeping the main code base behind the radiator open to anyone who might be curious to see how it works.

Before:

Trello.configure do |config|  
  config.developer_public_key = 'THE_DEVELOPER_KEY'
  config.member_token = 'THE_MEMBER_TOKEN'
end

After:

Trello.configure do |config|  
  config.developer_public_key = ENV['TRELLO_DEV_PUB_KEY']
  config.member_token = ENV['TRELLO_MEMBER_TOKEN']
end

Like the sound of how we work? Check out our Careers Page!

Testing auto_now DateTime Fields in Django

Django's auto_now_add and auto_now field arguments provide a convenient way to create a field which tracks when an object was created and last modified.

For example:

class BlogPost(models.Model):
      title   = models.CharField()
      author  = models.ForeignKey("author")
      body    = models.TextField()
      created = models.DateTimeField(auto_now_add=True)
      edited  = models.DateTimeField(auto_now=True)
      ...

Unfortunately, they can make writing unit tests which depend on these creation or modification times difficult, as there is no simple way to set these fields to a specific time for testing.

The problem

Although auto_now fields can be be changed in code, as they will update themselves afterwards with the present date and time, they can effectively never be set to another time for testing.

For example, if your Django-powered blog is set to prevent commenting on posts a month after it was last edited, you may wish to create a post object to test the block. The following example will not work:

def test_no_comment(self):
      blog_post = BlogPostFactory()

      blog_post.edited = datetime.now() - timedelta(days=60)
      # Django will replace this change with now()

      self.assertFalse(blog_post.can_comment())

Even changes to an auto_now field in a factory or using the update() function won't last; Django will still overwrite the change with the current time.

The easiest way to fix this for testing? Fake the current time.

The solution: Mock Time

The auto_now field uses django.utils.timezone.now to obtain the current time. We can mock.patch() this function to return a false time when the factory creates the object for testing:

import mock
   ...
   def test_no_comment(self):

      # make "now" 2 months ago
      testtime = datetime.now() - timedelta(days=60)

      with mock.patch('django.utils.timezone.now') as mock_now:
         mock_now.return_value = testtime

         blog_post = BlogPostFactory()

      # out of the with statement - now is now the real now
      self.assertFalse(blog_post.can_comment())

Once you need to return to the present, get out of the with statement and then you can test the long-ago-updated object in the present time.

Other Solutions

An alternative solution is to use a fixture instead; however fixtures should generally be avoided as they have to be manually updated as your models change and can lead to tests incorrectly passing or failing - see this blog post for more details.

Another alternative is to create your own version of save() for the object which can be overridden directly. However this requires more complex code than using mock.patch() - and all that extra code will end up in production, not in the test as in the example above.

Frontend Testing with PhantomJS CasperJS Mocha and Chai

Let’s face it. Front end testing, or in fact, any sort of testing that involves you (the developer/tester) manually going through each scenario can be a gruelling process. This post isn’t about the importance of front end testing, because it's 2015.

I’m going to write about testing the UI and simulating user actions in the browser using PhantomJS, CasperJS, Mocha and Chai.

Before I proceed any further, here is a brief introduction to what each framework/library does:

Mocha

Mocha is a feature-rich JavaScript test framework running on node.js and the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

Chai

Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.

PhantomJS

PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

CasperJS

CasperJS is an open source navigation scripting and testing utility written in Javascript for the PhantomJS WebKit headless browser and SlimerJS (Gecko). It eases the process of defining a full navigation scenario and provides useful high-level functions, methods and syntactic sugar.

CasperJS provides us with some really neat functions to work with. Particularly:

  • casper.start() : Configures and starts Casper
  • casper.waitFor(): Waits until a function returns true to process any next step.
  • casper.waitUntilVisible() : Waits until an element matching the provided selector expression is visible in the remote DOM to process a next step. Uses waitFor(). I find this particularly useful especially when we are interacting with DOM elements that have animation.
  • casper.capture(): Proxy method for PhantomJS’ WebPage#render. Adds a clipRect parameter for automatically setting page clipRect settings and reverts it back once done.

There are many more useful functions here.

CasperJS provides built in testing functionality as well, but we opted for the Mocha testing framework and Chai assertion library because we liked it better.

Let’s Get Started!

First get Mocha and Chai installed (if you opt to use Mocha and Chai instead of just using casper’s built in testing utility):

  • npm install mocha
  • npm install chai

Then install phantomJS: - npm install -g phantomjs

There are a couple of other ways to install phantomjs shown here.

Get casperjs running: - npm install -g casperjs

Finally get mocha-casperjs and casper-chai installed if you want to use Mocha and Chai with casperjs.

npm install -g mocha-casperjs

npm install -g casper-chai

Note from the future: I recommend installing the npm packages locally and specifying the test commands within package.json instead.

Alternatively, if you’re happy with just phantomjs and casperjs, using the Chrome extension tool here can help you get off to a good start. Resurrectio allows you to record a sequence of browser actions and produces the corresponding CasperJS script.

Ahem. Let’s really get started now.

Let’s write some code that does all of the following in sequence:

  1. Loads a page
  2. Waits for a selector to appear
  3. Clicks on an element and asserts that the redirected page has the correct url. If successful, capture a screenshot.

 describe('Home page', function() {
 	before(function() {
		casper.start('http://localhost:8000');
	});
	it('should have an element in DOM', function(){
		casper.waitForSelector('#correctElement', function() {
			'#correctElement'.should.be.inDOM;
		});
	});
	it('should bring you to another page on click', function() {
		casper.thenClick('#destroyEverything', function(){
			this.echo('Clicked on Destory everything');
		});
		casper.waitFor(function check() {
			return this.evaluate(function() {
				return /urlthatwewant/.test(document.location.pathname);
			});
		}, function then() {
			// Succeeded
			this.echo('->Succeeded in loading the another page');
			this.capture(anotherPage.png');
		}, function timeout() {
			this.echo('Failed to load page').exit();
		});
	});
});

Running the above is simple.

mocha-casperjs <filename>.js

Here are some things we learnt while setting up frontend tests for Kogan.com.

Keeping it DRY

Many of the functions can be reused. Fortunately, PhantomJS allows you to import/require modules using the CommonJS syntax.

A very reusable module would be the login module (simplified version) (as shown below)


module.exports = function(email, password) { 
  describe('Logging in', function() {
    it('Filling in form fields and clicking should log the user in', function() {
    	casper.then(function(){
    		this.fillSelectors('form.form-email', {
    			'input#email' : email,
    			'input#password' : password
    		});

  			this.echo('Filling in details');
  		});

      casper.thenClick('#loginButton', function() {
        this.capture('filledIn.png');
        this.echo('->Clicked on Login');
      });   

  	});

  }); 
};
Config File

Having a config file with all your valid/invalid data can also help with making things DRY-er.

	
      /**
      config file
      **/
      module.exports = {
      URL: {
      dev: ‘http://localhost:8000’
      }
      };
    

Putting them all together, we could make the code a lot cleaner, and easier to understand. Pretty sweet huh?


var config = require('./config');
var login = require('./functions/login);
var doSomething1 = require('./functions/doSomething1);
var doSomething2 = require('./functions/doSomething2);


describe('Yet Another Page', function() {
  before(function() {
    casper.start(config.URL.dev);
  });

  login();

  doSomething1();

  doSomething2(1);

});

Like the sound of how we work? Check out our Careers Page!

References: - PhantomJS - CasperJS - Mocha

Continuously Improving our Process - Retrospectives

Like many agile based teams we regularly run retrospectives to gauge as a team how we are going and think of what and how to improve.

We have one every two weeks, they are time boxed to one hour and are held standing up (like we do for nearly all of our team meetings).

We have the typical ‘happy’ column, a ‘sad’ column and a ‘puzzling’ column. Everyone brainstorms on post-its, we group it together and vote on what we would like to discuss.

The team is prompted by being next to our wall and running through some of the achievements/big events in the past two weeks.

I ask probing questions such as:

  • What has slowed down your progress?
  • What has enabled you?
  • From when you pick up a story to when you deploy it to production, what obstacles do you have?
  • What areas have been inefficient and where is there unnecessary work or rework?

There is one key part that distinguishes a productive retrospective from a time waster - the actions and improvements that are an output of the retro.

One thing that is tempting is to discuss the particular details of an issue that has just occurred - which can often turn into a bit of a whinge and sometimes even a blame game (especially if the people involved are not present at the retro). Then a tactical action is thought of to address the latest symptom and the team moves on to the next highest voted item.

It is far more valuable for the team to discuss the patterns and root cause of the issue.

To do this, the team should discuss what process was followed and give other examples of that process in play. This is to remove the emotion and raise it up to a more high level discussion about repeatable systemic issues.

Once we understand the root cause we explore what is within our control to improve. That way, when we begin to think of actions to improve the situation we are thinking of changes/tweaks to our process rather than a quick fix or bandaid.

Then, once we have an action to improve our process, we run through a few hypothetical situations to ensure we have a shared understanding of what our new improvement looks like. We’ll often look at a few tasks in our backlog to give us some examples and we run through how things will play out with our new improvement.

The next retrospective the first thing we discuss are our actions from the last retro:

  • Has the action been implemented? if not, why not?
  • Is the issue still occurring? if yes, why?
  • Did the action improve the issue? if not, why not?

By beginning our retrospective with the previously agreed actions, it reinforces to the team the purpose of our retrospectives, which is, to share examples of our teams anchors and engines, to discuss their root cause and to action improvements to our process.