Jon22: The Website of Jonathan Dobres

The books have closed on Child’s Play 2010, and this year’s total is a truly awe-inspiring 2.3 million dollars. With 2010 in the total, this means that cumulatively, Child’s Play has raised just shy of nine million dollars. Nine million dollars over the last eight years, every single cent of it helping to improve the life of a sick kid. If that’s not something to be proud of, I don’t know what is.

But I didn’t sit down in front of the computer today to talk to you about that. I do that enough. Instead, we’re going to talk about this sweet chart I made. Incidentally, at the time of this writing, Googling “sweet chart I made” returns this as the first result. Who am I to disagree?

Last year’s chart was put together with Numbers. I’m generally very happy with Numbers—certainly much happier than I ever was with the sluggish, bloated, obtuse mess that is Microsoft Excel—but the chart I produced last year has some problems. The spacing on the x-axis looks weird, and that’s a poor way to format a date anyway. Since the key shows the annual totals, it kind of defeats the point of the chart. And why did I go with a filled line chart? Because every year has many missing data points, and a filled chart was the only way to get Numbers to draw each year as a connected line.

This year’s chart was put together with R and ggplot2. Here’s what I like about it, and what I don’t.

What I Like

• R and ggplot2. I can’t recommend R highly enough. It’s fast, flexible, powerful, and oh yeah, free. I now use it for all my data analysis needs. I intentionally gave myself a hellish, badly formatted CSV file to work with here, just to see if R could beat it into shape. No sweat. As for ggplot2, it’s overkill for some situations, but a great solution for most. Maps, anyone?
• The date axis. It’s nicely labeled, with every major gridline representing exactly one week. Look closely, and you’ll see that the minor gridlines split the weeks into days.
• I ditched the legend, and instead placed year labels at the ends of their respective lines. Extra special audience challenge: do this in Excel without killing yourself after fifteen minutes.
• Cumulative total is computed on the fly and automatically added to the plot’s title.
• In fact, the whole plot is defined programmatically, even the year labels, so adding in 2011′s data should be a cinch.

What I Don’t Like

• There are too many colors on this thing. ggplot2 computes those colors by finding equally spaced points along the rainbow, so as more colors are added, the difference between them gets smaller. I’m using these colors to keep each line visually separate, but why? Do you need to see every data point of every year? One alternative would be to color in the current year and the previous year, and turn all others a shade of dark gray.
• The larger problem, though, is that this plot doesn’t serve much of a point. I don’t have enough data to get an accurate sense of how quickly Child’s Play accumulates funds. Look at where the lines start. The early years start at $0, but the spread runs up to nearly $500,000. Is that variability a reflection of larger corporate donations kicking off the fundraiser, or is it that Child’s Play runs year-round, and the charity is taking more money in during the non-holiday months of the year? In short, the only reliable data points in the plot are the totals, in which case a simple table could tell you just as much.

Still, it’s a fun exercise. I certainly learned a lot about R while working on this, and that’ll pay off in the future. Maybe I’ll tackle Boston’s weather data instead.

‘Twas the day before Christmas,
Or less than a day,
And Jon had just realized,
He hadn’t written about Child’s Play.

It’s that time of year again, folks. The year isn’t even over, and Child’s Play has already outdone last year’s impressive total. This without even counting the contributions from the more ambitious arms of the charity.

Back when I wrote about PAX East, I mentioned that the convention had helped me come to a new understanding about why I find Child’s Play so fulfilling, but that I was saving the disclosure of this revelation for a more appropriate time. Well, if not now, when? If not when, how? Is not them, us? Etcetera.

During the final Q&A session of PAX East, I decided to get up in front of the microphone and publicly express my thanks to Jerry and Mike for the charity. I’d been writing this little speech in my head for two days straight, but when my turn to speak finally came I was immediately overcome with emotion and rendered incapable of coherent thought. I managed to croak out a rough sketch of what I had wanted to say and then collapsed, exhausted, into my chair (or at least it felt that way). Luckily, a woman who spoke after me (featured in this episode of PATV) managed to say exactly what I had meant to say, to the word. She told a tale much like my own: the fearful surroundings of the hospital, the pains of modern medicine, the lack of certainty about what would happen next, and the sweet, joyous relief that she found in video games. In my case, it was 1988, and relief came in the form of a Nintendo Entertainment System. For this woman, it was an original Playstation.

In other words, we had each had the exact same experience. The only difference was a matter of years. I don’t know what mysterious force put a Nintendo in my hospital in 1988, nor do I know how this woman’s hospital got a Playstation. But they were there in our hours of need, and they made all the difference in the world. Child’s Play is a charity that allows me (and you) to be that mysterious force, to make a positive difference for a kid in his or her hour of need. For me, personally, this transcends charity. It becomes a truly karmic act.

Hospitals need everything from Play-Doh to Playstations, and the year isn’t out yet. Get cracking.

And before I forget, here’s this year’s chart-in-progress, this time made with R and ggplot.

I became a published author as of a few weeks ago! The paper is titled “Perceptual learning of oriented gratings as revealed by classification images”, which is…a mouthful. Don’t get me wrong, I’m very proud of this. Designing the experiment, collecting the data, and writing the paper all added up to a tremendous learning experience, and the final product is a solid piece of work. Still, it’s not exactly beach reading. So, herein and forthwith, a plain-English explanation of this thing I just published. Yes, you too can understand science!

Before you can really understand what this paper is about, you need to understand perceptual learning. I’ve talked about this before, but here’s another quick primer. Learning of any sort requires practice, whether the goal is to recite all 50 state capitals from memory, ride a unicycle, or perhaps most interestingly, do both at once. In these examples, learning involves the parts of your brain that handle memory, motor skills, or both. Likewise, practice can also change the parts of your brain responsible for vision. When you perform a difficult visual task again and again (like, say, a dentist looking for cavities or a hunter looking for deer), the neurons responsible for processing this visual information become more refined, better at representing the important aspects of the task. It’s not that you simply understand what you’re looking at in a different way (which would be a change in strategy), it’s that you are literally getting better at seeing. Perceptual learning can enable a person to detect tiny changes in an object’s position, make a person more sensitive to detecting motion, enhance contrast sensitivity, and many other things.

So how do we measure perceptual learning? Typically we’ll sit you down in a dark, quiet room and give you a difficult visual task to do. At first you probably won’t be very good, and your answers will be random. But after making thousands of these visual decisions over several days, you will get better at it, and we’ll be able to measure that improvement. Usually we’ll boil all these trials down to a simple summary, like percent correct. The point is that we’ll sit you down for an hour at a time and have you complete 1,000 trials of an experiment, and out of all that data we’ll extract perhaps a handful of useful numbers.

This very common approach comes with some limitations. First, it seems a bit wasteful to sit someone down for a full hour and have only a few useful numbers to show for it. More importantly, though, remember that I’m using your behavior (how well you do on the task) to draw conclusions about what’s going on in your brain. This is problematic, because while I can see that you’re learning the task, I can’t say what, exactly, is being learned. This is one of the major debates in the field. Are you getting better at the task because your brain is becoming more sensitive to the important parts of the task, or because your brain is getting better at filtering out the parts that don’t matter?

Suppose that instead of walking away with a handful of numbers, I was able to produce an image from your data, a picture of the mental template you were using as you performed the task. To get an idea of what I mean, look at these images from a 2004 paper by Kontsevich and Tyler, charmingly titled, “What makes Mona Lisa smile?” The researchers were interested in what aspects of the Mona Lisa’s face influence her famously ambiguous smile. To answer this question, Kontsevich and Tyler took a picture of the Mona Lisa and then added what we call “visual noise,” basically frames of TV static. As you can see from the black and white images, the random noise alters the Mona Lisa’s expression in various small ways. Participants in the study were simply asked to classify whether all these different Mona Lisas looked happy or sad. Once that was done, all of the noise that was classified as making the Mona Lisa “happy” could be averaged together and then laid back on top of Mona Lisa, producing a “happy” Mona Lisa (right color image). Ditto all the “sad” noise, producing a sad Mona (left color image). The basic message? The mystery of Mona Lisa is all in the mouth.

Kontsevich and Tyler managed to produce these classification images very efficiently by layering the noise on top of the picture. But if you wanted to produce a classification image of the Mona Lisa purely from noise, you would need tens of thousands of trials before you started to get something that looked like a woman’s face. This presents a problem for those of us who study perceptual learning, because we usually like to examine how perceptual learning takes shape over the course of a few days. Therefore, we need a way to produce a good classification image purely from noise, and out of one hour’s worth of data. That way we can examine how the image changes day to day.

That’s really the whole point of our paper here: can we find a way to make good classification images from very little data, and if so, can we then analyze the classification image to figure out what, exactly, is changing as perceptual learning occurs?

We decided to apply this concept to a pretty classic task in the perceptual learning field: the detection of an oriented grating in noise. Oriented grating is just another way of saying “tilted stripe”. An hour a day for 10 training days, we’ll have you look at stimuli that are either 100% noise, or a mixture of noise and grating. The grating is always tilted to the same angle, until day 11, when we rotate everything 90 degrees (we call this the transfer session, since we want to see if your learning transfers to a grating of a different angle). In the image below, you see, from left to right, the training grating, the transfer grating, an example of what pure noise looks like, and an example of a noise/grating mixture.

The trick to producing classification images from small amounts of data is to simplify your stimuli as much as possible. In most studies gratings look more like the image at the left, but in our study we’ve eliminated all shades of gray, and kept our stimuli very low-resolution. Each stimulus is just 16×16 pixels, but blown up to about the size of a Post-it note on the screen. This way, instead of each stimulus being composed of several thousand pixels and several hundred colors, ours have just 256 pixels and two colors.

And it works! Here are classification images from one of our subjects for Day 1, Day 10, and the Transfer Day (“T”). As you can see, the classification image is fairly indistinct on Day 1, shows a much clearer stripe pattern by Day 10, and then gets worse again when the orientation is changed for the Transfer session (all images have been rotated to the same orientation for ease of comparison). In case the effect still isn’t clear, the right column applies some extra filtering to the images in the left to enhance the effect. And remember, while all our stimuli used only black or white pixels, the classification images have shades of gray because they represent an averaging of all those stimuli.

Our next hurdle was how, exactly, to measure the “goodness” of each classification image. Our solution was to calculate the Pearson correlation between each classification image and the target grating. In other words, a fairly straightforward measure of how well these two pictures match up, with 1.0 being a perfect score. Once you have that, you can see how the correlations change over time. In our data, image quality clearly improves for about six days, and then levels off. When we change the orientation of the target grating, performance drops back to square one:

My more eagle-eyed readers might notice something ironic about all this. Did we just go through all this trouble to create a classification image, only to convert the image back to a simple number again? But wait, there’s more. What if I created a grating at every possible orientation, and then calculated how well a person’s classification image correlated with each of those? I end up with what we call a tuning function. Ordinarily you can only get those if you’re plugging wires into animals’ brains and directly measuring the activity of their neurons. But have a look:

The red line represents the tuning function for Day 10 classification images. See where the red line intersects with the line labeled “45º”? That’s the equivalent of the Day 10 data point in the previous figure. But because I’ve gone through the trouble of creating this classification image and measuring its correlation with 180 different gratings, I can see much more than a single data point. I can see that there’s a big improvement immediately around the orientation that was trained, but that this improvement rapidly drops off as you move farther from it. In fact, most of the red line is below zero, indicating that the fit between the classification image and these other gratings has actually decreased, or been inhibited.

Then there’s the blue line, which represents the tuning function for Transfer Day. Its spike is 90 degrees away from the red line’s, which you’d expect. But the spike is also smaller and broader, indicating that learning to detect one grating transfers imperfectly to others. Also of interest, you might notice that the blue line has a noticeable “bump” near the spike of the red line. This suggests that even when subjects are told to search for this new Transfer grating, they are using a mental template that is tuned for what they had been trained on previously.

Lastly, no two people are ever going to produce identical classification images, which makes the images useful for revealing individual differences and strategies. Early on in this study, we noticed that some subjects produced very nice-looking images and showed very clear learning trends. Others seemed to produce murkier images and flat or even reversed learning effects (they seemed to get worse over time, not better). What to do with subjects like this is always a thorny question. So, we split the subjects into two groups: learners and non-learners. We found, somewhat surprisingly, that the so-called “non-learners” actually started off with better images than the “learners,” but that they never seemed to improve beyond their starting point. Our analyses showed that the non-learners tended to focus on the center of the stimuli. Their classification images looked like bright blobs, not stripes. It’s a perceptual strategy that worked well for them initially, but failed to generate measurable learning. Meanwhile, the learners started off a bit worse, but seemed to incorporate more of the stimulus into their decisions, and thus produced better classification images. In other words, the non-learners were lazy, and we were able to see and quantify this thanks to the classification images.

That, ladies and gentlemen, is the story: an efficient way to pull a picture out of a person’s brain, and what that picture can tell us about how that person learns. In closing, I’d like to point out that this thing didn’t spring fully formed from our minds. It was accepted for publication as of October, but for three years before that it was a work in progress. More than that, it was one huge learning experience for me. How to program the stimuli, how to design the experiment so that subjects don’t fall asleep in the middle of it, how to detect a cheater, how to analyze the data, how to slice and dice the data, how to ask questions, how to get answers, how to write it up, and how to handle the revisions, all of this was a a learn-on-the-job deal. I’d particularly like to thank my advisor on this project, Aaron Seitz, for all his help and guidance. Oh, and should anyone want to look at the actual paper, the citation, one more time:

Dobres, J., & Seitz, A. R. (2010). Perceptual learning of oriented gratings as revealed by classification images.Journal of Vision, 10(13):8, 8–11, http://www.journalofvision.org/content/10/13/8, doi:10.1167/10.13.8

“We live in hard times, not end times.” –Jon Stewart, October 30th, 2010

Those are words to live by, folks, especially the day after the midterm election. As I process my disappointment with Tuesday’s results, my spirits are bolstered by the still-fresh memories of Saturday’s Rally to Restore Sanity and/or Fear, which I attended in person, in the heart of Washington. “Do we actually want to do this?” I asked the Southerner, a few months ago. “Book a flight and do it for real?”

“Yes,” said the Southerner. Within minutes, he engaged the roughly 90 percent of his brain that is dedicated to the transit systems of these United States and secured us remarkably cheap travel arrangements for an insane twenty-four hour itinerary in DC. I won’t bore you with a post about the grandeur of the Capital, the beauty of the White House by night, the joy of catching up with friends, or all the details of the Rally itself. Suffice to say that it was a pleasure to be among the 200,000 people gathered on the National Mall for what turned out to be a hilarious, inspiring three hours. Highlights included a benediction courtesy of Father Guido Sarducci, Sam Waterston’s reading of The Greatest Poem Ever Written, and most importantly, Jon Stewart’s Moment of Sincerity. If you watch one online video this year—or for the rest of your life, for that matter—let it be that one. Being there in person for that moment was, I believe, a once in a lifetime experience.

As we boarded the Saturday evening Amtrak back to Boston I was too tired to think much about the whys and wherefores of the trip. In fact, I was too tired to read, sign a check at dinner, or drink through a straw. Sleep deprivation can do incredible things to you. But now my mind is clear, and it’s time to put into words why I felt it necessary to push myself beyond exhaustion for the sake of three hours on the National Mall.

I’ve been watching The Daily Show every day for years, rarely, if ever, missing an episode. My roommates know that Monday through Thursday at 11:00PM is Jon Time. If I have a religion, Stewart and Colbert are my high priests. I had been watching the show off and on since 1996, before Stewart was the host, before the show even had a studio audience, and the only laughter on set was the sound of the show’s producers. I liked The Daily Show well enough, especially after Stewart brought his sharp intellect to the show, but I wasn’t yet the sort of fan who would travel to DC for a rally.

As it happens, I can tell you the exact moment of my conversion: October 16th, 2001. I had just started my freshman year at college, and America had just been attacked. I was on my own for the first time in my life in a strange city, and less than a week into this new stage of life, the entire world changed. The towers were brought crashing down, and with them, any notion that I lived in a safe society. Then came the letters laced with anthrax. This during an autumn in which everyone, everyone had a cold or a cough. I felt rootless, uncertain, and yes, frightened. It’s not that I wasn’t enjoying myself in college, or that I wasn’t able to get on with my life, but a dark cloud seemed to hang over everything. I couldn’t shake the feeling that another disaster was right around the corner.

I was in my dorm room watching a recently-posted clip of The Daily Show (this was pre-Youtube, and Comedy Central didn’t see much point in posting entire episodes then). Jon Stewart opened another of his “America Freaks Out” segments with a smile and said, “Alright. Think of a number between one and ten but don’t tell me.” He held his fingers to his head, pausing for effect. Opening his eyes, he looked into the camera and said, “Is it anthrax?”

And just like that, the clouds parted. For the first time in what felt like years, I laughed. Really, really laughed, not just at the joke, but at myself. With a single punchline, Stewart had shown me the heart of my anxiety, and how crazy it was. Could another attack happen? Could terrorists rain poison from the sky and kill us all? Sure. But it’s ridiculous to let the possibility fill every second thought in my head.

Jon Stewart restored my sanity that day, and The Daily Show has been mandatory viewing ever since. That is the reason I went to the rally, and the reason that the Moment of Sincerity resonated so strongly with me. I can only hope that we take that message to heart. When we are besieged by racists, extremists, demagogues, and a media that values drama over reason, when all seems hopeless and it looks like poison is going to rain from the sky any minute now, we must take a moment to reclaim our sanity. We must remember that we live in hard times, not end times.