An Inside Look with Codeship: Katrina Owen of Exercism.io

Katrina Owen is a developer and creator of Exercism.io, a code exercise, review, and mentorship tool for developers. This week, we had the chance to sit down with Katrina to discuss the human challenges of developing Exercism, learning new languages, and avoiding burnout in industry.

An Inside Look with Codeship is a regular series providing an insider’s perspective on founding and building a tech company. Each session, we chat with some of the most exciting voices in tech and ask them where they’ve been, where they’re going, and what we could all be doing together. You can read all Inside Look interviews here.

At the moment, my biggest challenge is shaping the user experience on exercism.io into something delightful. UX is something that I have no experience with. I’m not a full-stack developer; I fall squarely into the backend of web development, and I'm rarely faced with having to touch HTML or JavaScript. In my day job, I’ve never had to figure out what information needs to go where or how things should arranged or how one action should flow into the next in order to be understood by humans.

This is a huge problem on Exercism. The project started out as a cross between an experiment and a workflow optimization. There are people using it who've been programming for a couple of weeks and others who are retired. I never thought through who Exercism was really for, and more or less just slapped on a hack there and a patch there whenever someone complained about something.

I’ve only recently realized that the project bridges an important gap for people who are new to programming. In particular, it could be a really big deal for people who have just enough experience learning programming that they’ve decided to be serious about it and possibly make a career out of it. This is someone who has done tutorials and watched videos, and they’re trying to figure out how to go from where they are to being able to solve problems independently.

Even though Exercism is great for a lot of other types of developers, I think the committed newbie is the person who can benefit the most from it... but only if they’re able to get started using the site.

If you go to the website today, it’s incredibly difficult to figure out what the site is about, how it works, and whether or not it’s something that could benefit you. The onboarding process is haphazard, the site is full of jargon that early programmers probably don’t know, and the process itself is split between your local development environment, the command-line, and the website. It’s no wonder people are confused.

I’m working with several people right now to fix all of this, among them Jenn Dodd who is a phenomenal user experience designer. I think that over the next three or four months, the site will see a significant improvement.

In February 2013, I joined what is now Turing School of Software and Design to help work on curriculum and mentor the students during the six months of the developer training program. One of the tasks that I took on was to create little warm-up exercises for the students. Every morning we would give them an exercise, and they’d work on it for about 30 minutes before getting started with the real work for the day.

At first, I just gave the students a README with a problem description and expected them to run with it. It became obvious very quickly that they didn’t have the skills yet to break down the problems and write programs to solve them.

To make matters worse, even when they were able to write code that did what the README described, they never wrote automated tests. I figured if they weren’t writing unit tests for simple things like the warm-ups, then it was unlikely that they would have the skills to write tests for larger, more complicated projects. Since I saw testing as a crucial skill for someone who wants to do development professionally, this was a big red flag.

In the end, I decided to write the test suite for them. In part, this was to model good behavior, but it was even more important because the tests would serve as a guide to how you could solve the problem by breaking it into smaller pieces.

The tests helped a lot, but there were always some students who weren’t able to finish the problem within the 30 minute warm-up session. I assumed they would take it home and work on it or ask someone for help if they were stuck, but I was wrong. They basically never looked at unfinished exercises again.

Right before the first group of students was ready to graduate, I was helping a student with some code they were writing for a project and wondered out loud why they were doing a particular thing in such a non-idiomatic way. The warm-ups had addressed things like this over and over again, and they should know it by now. I realized that nobody had ever looked at their solutions. If we hadn’t talked about the fundamental concepts that come up, then students would have no reason to know how to solve these things idiomatically. As a result, students were writing code that was overly complicated and sometimes quite unreadable.

I had a lot of experience with the value of providing feedback on toy problems. Since late 2006, I had been participating as a volunteer on the JavaRanch Cattle Drive providing feedback via email on a more-or-less daily basis. We would talk about simplicity, naming, readability, and the rules of optimization:

1. Don’t.

2. Don’t... yet.

3. Profile first.

The results were dramatic improvements to the readability and simplicity of the code that people wrote in their day-jobs.

So about five months into working with students, I had three problems I was trying to solve. First, every morning at 8:45 I had to manually push an exercise up for the students. This is annoying and impractical. Second, the students were not finishing an exercise before moving on to the next one. Third, the students were missing out on some profound and very fundamental lessons because we weren’t providing feedback.

I decided to write the first prototype of exercism.io to solve these things.

For the past few years, my days have been almost exclusively about work: Get up, work on Exercism, go to work, come home, work on open source. On the weekends, I would do more work on Exercism or work on the book that I’m co-writing with Sandi Metz. About half the time, I would probably be working on a presentation for a conference.

A couple of weeks ago, I quit my day job so that I can focus more on the book and Exercism. In a few weeks, I’ll start picking up some contract work, but I intend to be more ruthless about cutting out certain types of busywork and being a lot more choosy about which obligations I take on. It turns out, I really value having time to gaze out the window.

Most of Exercism is very straight forward: Stick data in a database, take it back out. There is one piece that I’m incredibly delighted with, even though it is still in the experimental stages, and that is rikki- the friendly neighborhood robot.

Throughout the first year of working on Exercism, I spent a lot of time providing feedback to people, often on the same simple problems. Over and over, I saw people making the same type of mistake, and over and over, I would write up some variation on the same feedback.

To start addressing this, we implemented rikki- (a.k.a., Rikki Dash) who, being a robot, is perfectly suited to help humans understand machines.

We use some custom static analysis, written by JR Richardson to detect a small set of very specific issues that we had observed repeatedly. For each issue, we have prewritten a friendly critique. When someone submits a solution, and we detect violations, we’ll randomly pick one and have rikki- submit the corresponding comment.

At the moment, rikki- is only responding to issues on one particular exercise in Ruby. It would be fun to see if we could implement analyzers for some of the other languages as well and expand it to respond to a broader range of issues.

I’m not sure I had a very clear motivation when picking up Go. I met one of the developers working on the language at Google a couple times and was inspired to play around with it.

When Exercism started expanding and providing exercises in more languages, I had an extra impetus to pick up Go. It seemed wrong to provide a command-line tool that required a working Ruby environment if you were going to do exercises in Python or Haskell or Elixir. Reimplementing the CLI in Go let us distribute a binary that didn’t have any additional runtime requirements, and it was a cinch to compile it for all the platforms we were supporting. Even Raspberry Pi.

I think that as a backend developer, I found Go appealing because it lets you get a bit closer to the machine. I have a deep appreciation for consistency and simplicity, and Go gives you both of those things. And the tooling around Go is outstanding.

I like Ruby for all of its eccentricities, and I like Go for its solid, dependable simplicity. And its speed! In Ruby, a test suite that takes less than a minute is considered fast. In Go, you measure in nanoseconds, and a test suite that takes a whole minute… there’s likely something wrong.

I reach for Go if I need speed or concurrency. I reach for Ruby if I just need to write a quick one-off script to munge some data or experiment with the response from a third-party API. I still use Ruby a lot for Exercism since that was my primary language when I started working on it, and there are a lot of contributors who know the language well.

I don’t prefer one language over the other. They solve different problems. They say you should learn a new language every year, but it usually takes me about three years to get my head wrapped around a language. Maybe a year from now I’ll start tackling a purely functional language like Haskell or Erlang.

I am completely unqualified to answer this question.

The creators of Go spent years creating programming languages before they created Go, and they’ve spent years creating Go. There’s a lot of whining about certain features or lack thereof (cough, generics, cough) in the language, but I am impressed with how each feature in the language is there for a reason.

The creators of Go have made some very tough trade-offs, and they did so for reasons that are extremely well-articulated. They don’t think they have all the answers, and they don’t think the language is perfect, and they’re unwilling to compromise and put a half-assed feature in the language just because it would be nice to have or because someone wants it.

I’m sure there are a number of things that will be improved over time, but for the time being, I think that if Go doesn’t have the features you need to solve the problem at hand, then pick a different language.

I think the most interesting insight that I’ve had is that code is much less important than we programmers like to think. Product design, visual design, information design, user interface design, communication, collaboration, empathy -- these are all far more important than programming when it comes to solving the right problem and solving it right.

I approached Exercism as a purely technical problem: Write code that talks to an API. Munge data. Show the user some stuff. Make sure it all runs on a continuous integration server.

It turns out, Exercism isn’t much of a technical problem at all. It’s about providing a place where people can communicate within a very focused framework. This is a human problem. The technical piece makes it possible, but Exercism is completely uninteresting without the rich conversations.

There are so many human challenges in this project.

Exercism is open source, and that requires collaboration and coordination to a degree that I’ve never experienced before. For Exercism to really grow into a proper product, we have to look beyond just coordinating pull requests and issues. We need to think about higher-level questions about roadmaps and storytelling and logistics.

Becoming a contributor to the project is overwhelming. There are over 50 repositories, one for each language track, one for the CLI, another for shared metadata, one for rikki-... it goes on and on. It’s chaotic. To make matters worse, the contributor documentation is all over the map. If all you want to do is fix a typo, then you don’t want to have to wade through a bajillion contributor docs, you just want to know where to fix the typo.

We don’t really have a roadmap. People complain about little things and suggest solutions to them. For a while, I thought I had a thousand problems on Exercism, but it turns out, the individual suggestions are not what’s important -- it’s the fact that you have 30 people who are all talking about the same underlying problem. I don’t have a thousand problems, I have -- maybe -- seven. They’re big and meaty, but that’s a lot less intimidating than an endless stream of problems with no solutions in sight.

Even things like marketing have been completely overlooked. Anyone who heard about Exercism did so because someone got excited about it and shared it with them. It would be terrific if someone would be willing to contribute their writing skills and become the editor of the (non-existent) Exercism blog, and the (sorely neglected) Twitter feed. Open source is so much more than code.

We have over 20 different language tracks. Each track should be its own lively ecosystem with people who help improve the exercises so that they take advantage of the more interesting features of the language. Each ecosystem should have core members who delight in the language itself and enjoy evangelizing. These are the people who would provide feedback on the site daily. If we can develop this friendly, bustling environment where language enthusiasts can share their love of the language, then perhaps this can spill out into the real world into birds-of-a-feather sessions at conferences, meetups, hack days, and face-to-face mentorship.

Human contributions (storytelling, support, documentation, logistics, coordination) are hard to come by in open source. The code bit is exciting, but without the storytelling and the big-picture visionaries, open source is just a tedious list of bugs and issues.

I think that we’ve optimized for burnout. We work and work and work, and we almost never let up. We act as though working more hours is going to give us better results, even though there's a ton of research that suggests that working more hours causes our productivity to nose dive.

I wish that the industry would be more about taking breaks, staring at the ceiling, and just plain doing something else. I’m all for working hard, just not quite as relentlessly as we do.

I think we need to let our brain check out of the linear, task-oriented mode and flip into the mode that comes up with surprising solutions and non-obvious approaches. We don’t spend enough time connecting the dots between disparate things that we know. We acquire lots and lots of knowledge but develop very little wisdom.

I think we all could use a proper vacation.

You don’t have to know everything -- just solve the problem in front of you. Read the error message. Don’t just look for a magic incantation to make it stop failing. Make sure that you understand what went wrong.

Take your time. Go for a walk.