Holovaty.com

Back in the Django saddle

Adrian Holovaty — Mon, 28 Nov 2011 14:19:11 -0600

I've been a bad BDFL.

As a co-creator and one of the two Benevolent Dictators for Life of Django, my responsibility is to guide the open-source project forward. I did a great job of this for several years: from our open-sourcing in July 2005 (and for about two years prior to that), I was rather obsessed with it, contributing thousands of commits, answering thousands of support questions, participating in hundreds of design discussions, co-writing two editions of a book, writing week in review blog posts and evangelizing the framework at events. Making Django's code and community better is how I spent a significant chunk of my free time.

It grew wonderfully and brought me some of the proudest moments of my life, such as Google's launch of App Engine (with Django support built in!), a full-blown conference devoted to the framework and the consistently mind-blowing experience of being handed a book about Django written in a language other than English: Japanese! Chinese! Italian! Not to mention hearing about all of the incredible applications people around the world are building with the framework.

But then, in July 2007, EveryBlock happened. Running a startup took over my life, and even though EveryBlock is itself a rather complicated Django application, meaning I use the framework daily in non-trivial ways, my contributions to the framework itself basically stopped. And though I'm just one of more than a dozen core contributors, it's fair to say the project has suffered because of my absence. The development process got slower and too bureaucratic, contributors got frustrated by our lack of decision-making, and changes were made to the framework that I would've prevented had I been more involved.

I've felt horribly guilty about this for the past several years, and every few months I'd say to myself, "OK, I'm going to get back into Django development. For real this time." I'd fix a few bugs, make some documentation tweaks or whatever, but then I'd inevitably get distracted by my day job again.

But today, everything changes for the better. I'm excited to share the great news that Adrian's Lost Weekend is over. I've gotten the OK to spend a full workday a week on Django!

This is made possible by the growth at EveryBlock. I recently hired my own boss, after running the show for four years, and we've expanded our development team significantly. We do have some specific needs at EveryBlock that I'll be addressing in Django -- making speed improvements and fixing various pain points -- but I've made it clear that not everything I do on Weekly Django Day will be directly applicable to EveryBlock. Django, and my involvement therein, has yielded great dividends for EveryBlock, and it's time to formally give back.

The one-day-a-week regimen is exactly the kind of structure that will help me get back in the groove. I plan to work on these priorities, to start:

Release version 1.4.
Move to Git/GitHub. I want us to move from Subversion to Git, hosted on GitHub. Obviously, this is a large project with many repercussions, not the least of which is how our bug tracker and development process will change. We'll likely end up with some sort of hybrid system with our existing Trac installation, given how much we have invested in it. I predict some pain during this process, but it'll be worth doing for a whole bunch of reasons.
Reinstate "This Week in Django" blog posts. These were a great, concise way for Django users to stay updated with developments in the framework and community. I'm going to try to get these going again on the official Django blog.
Implement easy Pjax support. I have some design ideas on how we can easily add intelligent Pjax support to any Django app, with minimal code changes. I'm really excited about this one in particular.
Speed up template rendering. Armin has repeatedly talked about how much slower Django's template rendering is than Jinja's, and I want to cut out our inefficiencies to bridge the gap.
Fix transaction management. Not sure yet whether this only affects PostgreSQL, but I'm tired of the "idle in transaction" problem.
Fix the way .exclude() works. This one, described here, is getting in the weeds a bit, but it's an important thing to fix.

Look for more discussion on these issues on the django-developers mailing list. (My blog's comment section below is not the place to talk this stuff over. :-) )

I'm officially starting the one-day-a-week thing next week. It's good to be back. Let's get to work.

Introducing the YouTube Insult Generator

Adrian Holovaty — Sun, 16 Oct 2011 22:50:24 -0600

(It has been way too long since I launched a side project. Time to get back into the game. This blog post is adapted from a five-minute talk I gave today at Ignite Chicago.)

Everybody knows YouTube comments are atrocious. This is referenced all over the place, from a bunch of blogs to xkcd (twice!). One guy even took the time to make YouTube Comment Snob, a brilliant browser extension that hides poor quality comments.

If only we could extract some value from all that crap.

The YouTube community is so huge at this point that "meta" comments are very common. One particular "meta" comment stems from the fact that YouTube changed its rating system in early 2010. It used to use a five-star rating system but moved to a simpler "thumbs up" or "thumbs down" model, citing the fact that most people gave either one- or five-star ratings anyway.

In this new rating system, each video displays how many people have liked -- and disliked -- it. In a classic example of interfaces influencing behavior, this has encouraged users to make insulting remarks about the dislikers. I'm sure you've seen the type: it's comments like "439 people own a zune" on the 2001 Steve Jobs iPod presentation. (439 people had disliked the video at the time the comment was posted.)

After groaning about these types of comments for a while, I realized they're a sort of semistructured information -- it's always a number, followed by some sort of insult. And, besides, some of them are actually kind of witty. Hence my new project: The YouTube Insult Generator.

This is a basically a "search engine for insults." Type in a search term, and it'll give you insults you can use against a person who doesn't like that term.

For example, enter "the godfather," and it'll give you "You sleep with the fishes," "You sleeps with horsehead in bed" and "You will get an offer you can't refuse." Enter "alfred hitchcock" and it'll say "You had your eyes plucked out by crows" and "You have Vertigo." Enter "mario brothers" and it'll say "You aren't Super enough for Mario," "You can't beat world 1-1" and "You are bowser." You get the idea.

It finds stuff only about 50% of the time, but it works surprisingly well when it does work. Try general terms ("car") and pop culture ("michael jordan", "i love lucy"). Each insult includes a link to its source YouTube video.

How does this work? It uses the YouTube API to search for the top 50 most relevant videos for your search term. For each of those videos, it grabs the latest 50 comments. Then it looks through all that for comments starting with a number followed by a word such as "people," "youtubers" or "nincompoops." (View source for the full list, a regular expression that would make Alex Gaynor proud.) Finally, it just replaces the number and the word "people" with "You."

It doesn't do anything fancy like caching or giving users a way to mark searches as particularly funny, but it's not bad for a quick hack. Enjoy, and please use the insults wisely.

UPDATE: Wired wrote about it. Also, a few people have been confused about what the point of this is. It's not intended to be literal -- I mean, I don't expect people to use this, at face value, to create insults. The larger point is that it's a demonstration of finding structured data in unexpected places. I think we should experiment more with this sort of "poor man's data mining" on things like YouTube comments. Thanks for checking it out!

Crash tags

Adrian Holovaty — Thu, 14 Jul 2011 23:38:18 -0600

Chris Messina, the guy who started Twitter hashtags, proposed the use of hashtags on Google Plus earlier this evening.

There's some major cognitive dissonance going on here, seeing Twitter hashtags on Google Plus.

It really hits you over the head with how limiting (and, frankly, silly) Twitter is. I think in a few years people will look back at Twitter and feel like they were, well, duped in a way -- millions of grown men and women, adults, typing in obscure abbreviations and making up messy syntaxes just to fit a semi-arbitrary character limit; large, important companies falling over themselves to play the game and Build Their Brands around these tiny messages; opportunists positioning themselves as gurus of this Sophisticated New Communication Platform™.

It feels like the Twitter founders are sitting in a room somewhere, laughing at how they've gotten millions of otherwise-sane people to jump through these absurd hoops -- and that, deep down, even the most guru-y of the Social Media Gurus ask themselves from time to time, "Seriously, I'm doing this?"

Anyway, it feels like G+ should have something much more sophisticated and user-friendly.

Job opportunity: Web/mobile developer at EveryBlock

Adrian Holovaty — Wed, 06 Jul 2011 13:56:42 -0600

We're looking to hire another programmer to join our small and effective development team at EveryBlock. This is your chance to work on a wide variety of interesting problems, helping improve a great product with a passionate and growing user base. You should expect to have a hand in all aspects of the site, and your contributions will have an immediate, direct impact on the awesomeness of our service.

When it comes to developers, we like to hire Jacks (or Jills) of all trades. The ideal candidate will be equally comfortable developing back-end Python code, working with databases (we use PostgreSQL and PostGIS), writing efficient front-end JavaScript (we use jQuery), writing screen-scrapers and other various data-import scripts (we use Python), helping optimize our infrastructure, and working on EveryBlock mobile applications (iPhone, Android). There will be a focus on mobile applications, at least at the beginning, but your responsibilities will be wider and more varied than that over time.

Here is the requisite bulleted list of requirements:

Significant experience using Python.
Experience building mobile applications for iPhone and/or Android.
Experience using Django to build Web applications.
Experience developing non-trivial JavaScript applications.
Experience deploying code on Linux.
Experience (and preference for) working on a small team.
Clear communication skills, both verbal and written.
Impeccable work ethic and ability to manage your own workload effectively.
Ability to iterate quickly.
Ability to write high-quality code, with a passion to the point where poorly written code makes you slightly nauseous.
A passion for computer programming/hacking to the point where you'd be doing it even if you weren't getting paid for it.

And here are some nice-to-haves:

Experience building geo apps and using PostGIS.
Experience writing screen-scrapers and working with data (cleaning it up, importing it).
Passion about improving neighborhoods through information and enabling community conversation.
Sense of humor.

This is your chance to work directly with two Django committers, including one of the creators of the framework (hey, that's me!). We have three developers at the moment; you'd be the fourth. It's worth mentioning that the entire team right now is seven people, and you have a chance to make a huge impact, beyond strictly development -- suggesting marketing ideas, helping prioritize various strategies and tactics, etc.

We have a low-stress environment and a culture of getting things done with as little corporate BS as possible. Though we're technically no longer an independent startup -- we were acquired by msnbc.com in 2009 -- we're culturally very much still a startup, given that our product is still in early stages and we're still figuring things out. It's a very nice combination of startup culture with the financial security of working for a big company. (We have great benefits -- salary, health care, 401(k), bonuses, etc.) And msnbc.com is taking the long-view on us, investing in us over time and giving us years to develop a large audience and become profitable. It's a great company to work for.

Now is a fantastic time to be joining our team. We just hired a new President, we recently launched a major redesign that's been almost universally loved, and, most importantly, we're getting momentum and critical mass in more and more neighborhoods.

You'll need to live in the Chicago area for this position; telecommuters need not apply. Our office is in a comfortable loft space on the north side of Chicago, near a bunch of other startups. Several of us walk or bike to work; we're also equidistant from the Montrose and Irving Park el stations on the Brown Line, and several bus lines.

Apply through the official msnbc.com Jobvite site here. Thanks for checking this out.

Wanted: EveryBlock general manager

Adrian Holovaty — Tue, 08 Mar 2011 17:00:59 -0600

As EveryBlock has grown, I've started realizing two things:

We're at a point where we could really benefit from a strong, business-minded person who can take our company "to the next level."
Personally, I'm much happier, and frankly more useful, focusing my efforts on the product than the business end of things.

So I pitched the idea of hiring a "CEO" type person to my msnbc.com colleagues (who own EveryBlock), and to my delight, they were fully supportive of the idea. We just posted a job ad for this new position, which we're calling General Manager; you can read it here. It's based in Chicago with (most of) our team, and I hope you'll pass it along to anybody you think is qualified.

My role will shift to focus on the product instead of running the whole show -- which is essentially what it's been for a while now, but we're making that more official.

On a separate note, I can't believe it's been more than a year since I've posted here. Stay tuned for a backlog of projects, plus some big EveryBlock product-related news. :-)