In March, we built the ground-breaking SXSW Zeitgeist for Pepsico. It’s no longer running—SXSW is over, natch—but there are some videos, in case you didn’t see it.

Pepsi wanted to “do something cool around Twitter/SXSW” and we delivered the concept, design and execution. (And boots-on-the-ground troubleshooting.)

It was a big hit. It got mentioned in a number of places, including Brandweek and VentureBeat, and of course it was all over Twitter.

Fast forward to… now. We put the Zeitgeist thru a visual overhaul & made some technical changes, & now the Zeitgeist is riding again for Internet Week New York, once again very generously sponsored by Pepsico.

See the Pepsico Zeitgeist live

I wanted to write a little about what makes the Zeitgeist unique, and why and how we made it that way.

Groundwork

If genius is 99% perspiration, we’re an Einstein-Mozart mashup. Or just covered in sweat. Take your pick.

We started these projects with… you guessed it… sweating. I mean, research.

Who’s going?

We use a combination of keywords and stop words to attempt to identify people who are going to attend the conferences (as opposed to just talking about it, or worse yet, expressing regrets that they couldn’t come).

We come by these the old-fashioned way: the scientific method of hypothesis, trial, error, and revision.

What are they doing?

Then we follow them with our small army of benevolent Twitter bots.

We wanted to see what people were really doing, and talking about—flying, landing, getting taxis & hotels, getting badges, etc., etc.

You can’t do that if you rely on searches. Searches are just a tiny window into the whole experience of a person. They don’t show you what the tweets are on either side.

Timeliness

We changed which topics we selected for / displayed on a daily basis to showcase the conf experience people were actually having that day.

For example, for SXSW, we’d focus on names of big-ticket keynoters, and for Internet Week, on one of the many separate confs that were part of the bigger event.

We follow because hashtags suck

The hash tag usage at SXSW was extremely minimal.

The more fun you’re having, the more mobile you are, the less likely you are to drunkenly poke your iPhone to add “# s x s w” to your tweet.

Not that there’s an agreed-upon tag, generally, to add to the confusion.

By including an attendee in our bots’ friend timeline, we get all their updates, and can then pick & choose from there. We get information and updates that would never show up with a hashtag.

Hash or not? We’ve learned some interesting things about hashtag adoption and usage rates. Hashtag usage at Internet Week NY is much more regular, seemingly because: the demographics are older; people are attending for work; the demographics are more in the marketing/advertising world; these people are tweeting from laptops; they are less mobile, and less drunk, because IWNY is loose grouping of smaller, simultaneous conferences that have single all-day tracks, and few official events run late into the night.

Context-sensitive

The Zeitgeists are hand-tuned for each event, to squeeze out the best possible result.

For SXSW, we hand-picked upwards of 100 locations around the conference where we knew people would go, where parties would be, where people love to eat and meet up, dance and get sloshed. We came up with keywords that people were likely to use: not just Moonshine Bar & Grill, but just plain old Moonshine. Let’s face it, people are lazy.

And they will misspell Buffalo Billiards after the first few margaritas.

For Internet Week, we do select for words that indicate locations for the geographically spread-out events, even unofficial ones. And we update the keywords list every day for what’s going on that day.

Where they are, or where they’re going?

Through keywords, we identified where people were talking about, rather than attempting to geolocate them. What they’re talking about signals intent (e.g. “headin 2 buffalo billiards for the rails meetup!”) vs simply where they happen to be(“on the corner of 6th and whatever”).

How people move—or don’t move

For SXSW, we built the animated map, because people move around a lot in the downtown area around the conference center.

For Internet Week, because there was no single unifying event, and the city is so large, we skipped the maps.

Judging by the tweets we’re getting, this was the exactly right move. These folks aren’t partiers and roamers in the way that the SXSW attendees are.

What people really care about

It seems like more people attend SXSW for the parties than the talks. That’s cool with us. We are all about giving the people what they want.

We built a special Party Watch, to monitor tweets about the parties and the party venues during the time around the parties.

On top of that, we did some rough & ready natural language processing to find—and share—the answers to such questions as:

1. Does the Facebook party suck?
2. Is this or that party empty?
3. Where is there a line? Where is there no line?

(Not that we needed much to tell you that the Facebook party almost always sucks. And our tuned app reflected that.)

And we tuned it based on the topics, too. I’ve attended SXSW for 3 years running, so I’ve got the feel of the event.

Rather than doing charts & graphs of tweet volume, for instance, we focused on the things people do: meet, greet, attend, drink, eat, watch shows, and travel.

The secret to our success

The phrase “by hand” is our watchword.

To squeeze the absolute best data, you can’t rely on automation. It’s just that simple.

But tweets are small, short, and knowable. People tweet in predictable ways… and people who attend one conference behave, en masse, differently than people who attend a different conference.

There’s a distinct flavor difference, and we did hours of research to ferret that out.

Our keyword lists are huge. We tuned them. We unfollowed or followed people manually. We massaged the bots.

We even got a database of the SXSW tweets from last year and ran simulations on them.

It’s a lot of work, but it’s worth it. Our visualizers are top-notch.

Compelling design with awesome JavaScript effects

Thomas, of course, is the author of Scriptaculous and a JavaScript god. I’m definitely an expert, but it hardly bears mentioning when I’m in the same room as my husband.

With our powers combined, we took:

• carefully chosen people
• the information/insights that people are truly interested in

and made them into surprising, and mesmerizing, with animation, transitions, and effects.

We also tried to add elasticity and fun to the effects, not just cold, hard, mathematical exactness. And we tried to use them to support interest, mystery, suspense, and playfulness.

Tag cloud, schmag schmoud

For example, we didn’t just make a tag cloud.

No, we took the top words—just from our pool of tweeting attendees, mind—and dropped them onto the screen, all the same size. We created suspense, because you don’t know which ones are going to be popular. Then they explode up to their relative sizes, tag-cloud style.

Then they fly around.

Then, after they’ve flown around for a while, they convert into the sexiest stacked bar chart. Then that disappears and the tags drop down again.

One of the major issues with tag clouds is that they’re useless. They have no scale. How many tweets is 48pt type? With our version, you know, because the bar chart shows the exact number of tweets and the proportion.

It took a while to arrive at that design, and it took a while to develop, but it was worth it.

Moving pictures

We came up with two entirely new screens for Internet Week: one that shows just the tweets of their Twitter street team, and another that pulls related pix from Flickr. They look like this:

Animated bar charts are the awesome

And on one of the other screens, the topics (SXSW only) on the left convert to an animated bar chart. We have a video of this particular effect.

Bottom line…

People didn’t believe that it was JavaScript.

People assumed we were using hashtags.

But it is, and we aren’t.

So I thought I’d share.

Check out the videos and see it for yourself.

And if you’re interested in JavaScript-driven web apps, snazzy visual fx, and generally confusing people into thinking your site is Flash—but oh-so-much better—you should buy our JavaScript Performance Rocks! book while it’s still in beta. We cover everything from The Most Ridiculous Performance Fix Ever (and it *is* ridiculous), to serving strategies, DOM diets, loop unrolling (for those really extreme cases), how to play nice with the garbage collector, and everything in between. And it comes with our custom profiling tool, The DOM Monster, which analyzes your pages and suggests fixes. This package is only $24 now but will be $29 as soon as it’s done… but if you snag your copy today, you get the final version by email just as soon as it’s ready.