Jay’s Not Here

The great thing about living in an area as diverse as the GTA is that you end up with pockets of microcivilizations.

I was reliving my high school years loitering at a grocery store with a friend when I stumbled across this sign:

Yes, only one aisle out of about a dozen had 西人食品; i.e. Western person food.

Anyway, that vinegar, lemon juice and canola oil on the opposite aisle sure looks Canadian to me, eh?

Following up on my recent post about cognitive decline, I recently read about Beth Adelson’s psychological experiment[1] from 1984 in which participants were given a flow chart of a program written in a toy Algol-like language, shown the actual source code, then asked to recall details of the logic.

They found that novices’ error rate increased when they worked with more high-level flow charts and experts’ error rate increased when they worked with more low-level flow charts. So novices beat experts at concrete tasks and experts beat novices at abstract tasks.

This is interesting because it can invalidate Saltwood’s[2] general claim that people reach their peak cognitive abilities between the ages of 22-27. Why? The 12 psychological experiments Saltwood conducted to measure cognitive ability aren’t really representative of the complex tasks programmers would typically perform. The 12 were grouped into these four categories: reasoning, memory, spatial visualization, and speed. The only tasks that involved high-level reasoning were various types of pattern completion.

It’s probably more appropriate to say that people reach their peak for particular cognitive abilities between the ages of 22-27. But Adelson’s study seems to suggest that people get better (or worse) at certain types of reasoning based on their level of expertise (which usually comes with age).

[1] Beth Adelson. When novices surpass experts: The difficulty of a task may increase with expertise. Journal of Experimental Psychology: Learning, Memory, and Cognition. Vol 10(3), Jul 1984, 483-495.

[2] T. A. Salthouse. When does age-related cognitive decline begin? Neurobiology of Aging, 30(4):507 – 514, 2009.

Yesterday during cookie break, Jon and I were trying to figure out how we can measure the level of software development expertise of an individual without taking up so much of their time that they’d likely drop out of the study for which we originally recruited them.

We’re trying to figure out a set of indicators or tests we could use to figure out where on the novice-expert spectrum people fit. It seems like such tests would have to be context specific. Measuring the expertise of a web app developer would be much different than measuring a Windows desktop app developer because the domains and development methodologies are completely different. Assuming we could make such tests, would it make sense to compare the results of developers from different domains?

I’ll post more concrete thoughts about that in a future post, but any ideas in the meantime are certainly welcome

…I don’t usually check my Twitter account.

Is age a confounding factor in studies involving novices and experts?

This study[2] suggests that people on average reach their peak cognitive abilities between the ages of 22-27.

If that’s really the case, what does that mean for all the comparative literature about novices and experts?

Experts usually have many more years of experience than novices—usually enough to push them well beyond 27. The novices in those studies are usually undergraduate or graduate students. That could mean people as young as 17 or as old as—

…wait; how long does it take to finish a PhD, again?

[1] A song I played in stage band when my cognitive abilities were still on the rise; first time I ever ran across blastissimo; apparently, Google knows nothing of it, which isn’t surprising since the score I had was hand written…

[2] T. A. Salthouse. When does age-related cognitive decline begin? Neurobiology of Aging, 30(4):507 – 514, 2009.

When I was a lowly intern at OTI way back in 1999, my supervisor asked me to design and develop a plugin for their brand new extensible Integrated Development Environment called Eclipse.

Every term (by the university calendar), they’d get a batch of a dozen or so interns. Each one would have to write a plugin within the span of 2 weeks. I haven’t heard of any instances where interns failed to produce one.

Fast forward to today: Most university students I run into find it exceedingly difficult to get started writing an Eclipse plugin. What changed?

The code base has certainly evolved since the 1.0 release. However, the amount of documentation and sample code has also increased tremendously.

More importantly, at OTI, we had access to a significant level of resources. We had slide decks describing the architecture and design philosophy of the system. We had tutorials about how to use the tools to do things the Eclipse way. We had physical access to 90% of the people who wrote the code we were extending. (The other 10% were the Java tooling folks; whenever we asked the local senior developers where they sat, they’d tell us to go down the main hall, past the receptionist, turn left onto the street, wait for a bus to the airport and take the first plane to Amsterdam)

We drank their Kool-Aid from a fire hose.

Availability of resources has a significant effect on how quickly novices and experts familiarize themselves with a software system. This is obvious.

What isn’t obvious is what people actually do when these resources are constrained. What do novices do when they want to contribute to an open source project (say, for example, Google’s Summer of Code)? How do they get started? Mentors prefer students that are already familiar with the code base because that demonstrates their independence and potential for getting their project finished. How do students get themselves to that point? What works? What doesn’t?

I think there are differences in the strategies novices and experts use to familiarize themselves with a software system. One way of categorizing the various cases is to group them by expertise and the level of resource support they have:

…where Artifacts Only means you can dig through code, documentation, diagrams and the like but you’re investigating the problem on your own; Other Learners means you have access to other people trying to investigate the same thing but they may be just as lost; Artifact Creators means you can ask questions of the people who made the artifacts and (hopefully) get enlightening answers.

I’m currently digging through psychology and software engineering research to see where the various studies fit into this grid. So far, most of the research I’ve seen falls under N3 and E3. I’ve yet to find empirical studies that fall under the rest. I’ve come up with some examples for each of the other categories:

• N1: GSOC participants
• N2: University students performing change tasks on large software systems
• E1: Eclipse Bug Day participants
• E2: Teams that inherit code bases from other teams that have been dissolved (with little or no opportunity for knowledge transfer)

What’s the next image in the following sequence?

1
2
3
4	?

Here’s a hint—the three images are:

I’ve spent about $750 over the past year on flats. 280 days until I’m living car-free along the subway line—just a little longer…

Here’s the play-by-play:

Backing up your data is a good thing. Having your backup system die is not.

After being burned by data loss quite recently, I’ve started doing regular backups of the various systems I manage. It’s been easiest with my OSX machines because of this nifty little feature called Time Machine.

Unfortunately, Time Machine isn’t immune to hardware failure. And that’s exactly what happened. I just put in an order for a Time Capsule. I hope my data stays intact until it arrives.

Soon, I’ll have to do backups of my backup system.

If anyone knows of a good, free, simple and solid automated backup system for Linux (no Amanda or Bacula, please) or Windows, please let me know.