Kistaro Windrider, Reptillian Situation Assessor (kistaro) wrote,
Kistaro Windrider, Reptillian Situation Assessor

  • Mood:
  • Music:

I just love it when a plan comes together

The last two days at work have been pretty darn epic, at least from my perspective. I went from a general idea for what I wanted to write, to a design, to a first-pass implementation; heavy testing begins Monday.

I'm not sure why this should be such a big deal. It was, in the end, as simple a thing as I thought it would be (although it sounded sort of scary: take a parallelization framework designed to run a program with different parameters on different machines, then aggregate the results, and make it transparently distribute calculation of a parallelizable function inside the middle of a Python script, while completely hiding the complexity of building a task list, taking your parameters from the command line, or even going through a separate program launch at all), and I implemented the first pass at it in exactly the time and space I thought: 200 lines (ignoring comments), in two days. I'm still surprised at how short it is, for what it does; Python is like that, though. (Incredibly expressive, at the cost of speed compared to non-dynamic languages.)

So now the team of computational biologists can just write calculation scripts that just do their thing and distribute them across our large overpowered number-crunching computer cluster without worrying about any of the complexity involved in actually distributing the jobs. Frankly, my co-worker's job of writing the parallelization platform is more impressive (although the Go language gives it a similar degree of "astonishing elegance" as the Python code I'm working with), but I have now made it (okay, without testing done, "am in the process of making it", it's got a week to go) truly convenient to use.

What gets me, though, is something that eludes me to describe. I'm describing what was produced, the objective and concrete things that are; in the end, though, this felt cool because of what it was like. With exceptions for "ack, brain is full, need break", the code just happened. There was a next line to write, I didn't get lost in it, my design worked and fit reasonably well with what I actually implemented, it just came together. And I think the reason why this is cool is that it's not an outlier. Not anymore, anyway. This is how I did some pretty awesome stuff in college, but it sort of fell away at Microsoft. At my current employer, it's what writing software is like- so when I get to actually do that, it tends to be this successful thing I get to take pride in, and I'm still not used to that yet- but the idea that "this is how it is" has made me start to feel like I might be good at what I do, and that's a new experience for me.

I've migrated to DreamWidth. The original post is at View comments at; go ahead and use OpenID to post your own, or you can comment here.
Tags: geek, work

  • Last LJ post

    Hey all, I joined the LJ exodus train before it was cool</hipster>, but with recent developments in LiveJournal server location (…

  • (no subject)

    I want to assemble things that nobody else could ever assemble, and when they are done, I want to have done it in ways that nobody of average skill…

  • Failing, etc.

    That feeling of being 99% sure a social space would have been better for everyone without you in it, but you can't apologize or talk about it or…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 1 comment