Archives for August 2012

From time to time, we are going to have guest posts on this blog and today developer Adam Parkin from Victoria’s Seeker Solutions writes about how the Seeker team attended the inaugural Vancouver Polyglot {UN}Conference earlier this summer.

Seeker Solutions is a research-driven organization with a focus on content analysis using semantic technologies such as Natural Language Processing and Machine Learning. Founded in 2006 by CEO Duncan MacRae and based in Victoria, BC (although they have an office in Vancouver now), Seeker Solutions has identified how to use natural language information retrieval to access information in a more effective way.

 

 

 

 

 

 

 

 

Adam writes:

What’s a Polyglot? According to Wikipedia, Polyglotism is “the ability to speak several languages with a high degree of proficiency”.

What’s a {UN}Conference? A {UN}Conference is a unique participant-driven forum that’s well-suited to coding topics. Attendees suggest topics on whatever they want to talk or hear about via open forum. It’s a very cool idea that has the potential to be either completely awful or absolutely amazing.

Given this, the Polyglot {UN}Conference was exactly as you’d expect: a day-long open forum where a bunch of computer geeks get together and talk about pretty much anything programming-related.

There were a series of semi-organized discussion forums. Some were fishbowl discussions; some were presentations of cool technologies followed by audience discussion of related ideas. Folks from Seeker participated in a number of sessions. Here is an overview of one of the more interesting talks, as well as an outline of some of the other topics.

Testing, Testing, 1-2-3: Effective Testing Practices

The purpose of this session was to discuss testing practices in various development shops. There were several questions that the facilitator wanted to discuss, including: do development shops use both acceptance testing and unit testing? What is a ‘good’ coverage metric? One of the interesting threads that came out of the discussion was the use of a tool called Cucumber, which is used to specify tests in what is called Behavioural Driven Development (BDD). Cucumber uses a set of English verbs and nouns to describe scenarios that are meaningful to the end user with respect to the developed software. These nouns and verbs are then tied to various functions in the system. For example (using Wikipedia), a feature definition of “division” for a calculator may look something like:

Feature: Division
Scenario: regular numbers
* I have pressed 3 into the calculator
* I press divide
* I have pressed 2 into the calculator
* I press equals
Result: should be 1.5 on the screen

Cucumber looks extremely interesting, since its purpose is to test entire execution paths that are of interest to the end user. Given that it uses English to describe the desired execution path, the tests are extremely accessible to the end user, and could even be written by someone with little programming experience. On the back end, those English-like statements are translated into functions to be executed for the test, and the results are compared to what is expected. This instigated much insightful and spirited debate about the validity of having “non-programmers” use a system that’s like programming. Then the fire alarm went off and we all had to leave for a bit. Flame war indeed!

In any case, the discussion noted that unit tests should not be abandoned in favour of acceptance testing. In fact, a solid description was presented of the amount and types of tests to be performed, as follows:

/
/ – GUI tests
/—-
/ – Acceptance tests
/——–
/ – Unit tests
————–

The key takeaway from the above pyramid is that unit tests should be the most extensive, aiming to cover most of the functionality of the system, followed by acceptance tests and then finally GUI tests. Note that the difficulty of writing the tests increases the higher you are in the pyramid.

The discussion then shifted towards development practices that other shops were currently using. One person mentioned a presentation he had seen at another conference where it was suggested that unit tests should be thrown out after completion of the main development cycle, since they are an expensive “burden” to maintain. Many countered this fact, saying that unit tests can become really good regression tests. Some shops perform no testing at all, shipping out the product to a few relatively small clients until the bugs in the new release have been found and fixed. Additionally, some shops will immediately roll a system back to a known good state in the event of a bug, allowing developers to fix it before rolling it back out again.

Finally, an insightful discussion ensued, largely surrounding the notion of test coverage: trying to identify what constitutes “good” coverage, or if per-line coverage has any meaning whatsoever. Some feel that the artificial number of per-line test coverage could potentially give a false sense of security. Others feel that while the number is a metric just as meaningless for testing as counting lines-of-code (LOC) is to complexity, it does provide a quick-and-dirty metric for identifying gross shortcomings in testing. While there were few suggestions as to what makes a unit test ‘good’, the general consensus is that coverage can at least provide a rough idea of whether you are actually exercising the code in the test. The problem of whether you are actually testing your assertions was left open.
There were also some rather humourous “horror stories” about testing (or a lack thereof) in an industry, as well as discussion that started to touch on the deep issue of why we test. It’s unfortunate that this session lost 10-15 minutes (due to the fire alarm), as it really was an outstanding and insightful discussion.

Several other topics were touched on in other presentations and discussions throughout the day, such as:
• The Go programming language
• How to attract developers to your platform
• “Modern” concurrency mechanisms
• Big Data
• Javascript GUI frameworks (STOP THE INSANITY!)
• DevOps
• Fast, beautiful maps
• Python and the R language
• The CoffeeScript language
• From Consulting to Product Building
• Advanced Javascript

In addition to all of this, there were unstructured “breakout” talks throughout the day on whatever people wanted to discuss. Also available were three different day-long tutorials on Erlang, Git, and building a web server from scratch the day before the conference, as well as a day-long hackathon facilitated by Mozilla the day after. There was truly something for everyone.

Overall, the conference was quite well run, considering it was the first one of its kind. It had incredible value, too: $35 for a day’s worth of tech talks with people who not only know and love technology, but use it to solve problems on a daily basis. (The day-long tutorials were a separate fee, with cost dependent upon the tutorial.) Schedule permitting, I would love to attend this conference again in the future.