Gordon Bell, principal researcher at the Microsoft Research Silicon Valley Laboratory, talks with Techonomy founder David Kirkpatrick at Techonomy 2012 about his work on “lifelogging.” For edited highlights of this conversation, click here. (Transcription by www.realtimetranscription.com.)
Kirkpatrick: I haven’t even read my script here. But Gordon Bell, who is a living legend, please bring your legendary status out on stage.
Bell: Thank you.
Kirkpatrick: Gordon has done so many things in the technology industry and shown an uncanny ability to stay in the present. Not only did he play a huge role in the invention of the VAX computer at Digital Equipment Corp., which I’m old enough to remember and even used at one of my first jobs, but he has evolved throughout and he joined Microsoft a few years ago.
In—one of the other things he did, he worked at the National Science Foundation and played a major role in the development of the early Internet itself.
But in his last decade or so at Microsoft—how many years have you been at Microsoft?
Bell: Since 1995, so 17.
Kirkpatrick: He’s done tremendous work in Microsoft research. But particularly in the last decade, he’s really focused in on this issue of how we as individuals take advantage of technology and really in a way, make real some of Ray Kurzweil’s predictions about the merger of man and machine. Ray is somebody he knows fairly well.
I want to ask you, as a way of getting at that, to describe the device that’s hanging around your neck, Gordon, and then we can maybe get into what you’ve been up to.
Bell: This is a thing called a SenseCam. And it was invented or developed by Lyndsay Williams, who was at Microsoft Research in Cambridge. And I’ll say she was an inventor, and had the idea of having this camera that she’d wear all the time so that you would be able to say where did I put my glasses or my keys or something like that. Or if she had an accident on a bicycle in Cambridge, she wanted to record that. And it ended up being used. It’s licensed to a company in the UK.
And then it’s also a company here that was part of our session on lifelogging, Memoto is building, which is tech—which is being used for the tech start—
Kirkpatrick: Raising money on Kickstarter.
Bell: On Kickstarter. She’s got $500,000, 2,300 people who want a camera like this.
Kirkpatrick: How often does that take a picture?
Bell: Every 20 seconds. Then you upload the series, so you get this sort of frames of what you did during the day. Of course I look at it as a software sponge, because it takes so much software to get to do all of the things you would like to do. Every time you do something, you would like to do something else.
Kirkpatrick: You wore a device like that for several years, correct?
Bell: It isn’t fair to say—I wore it for a couple of hundred sequences. So I used it over about 200 sequences. And then really found that the limit—so I would use it for recording events. I mean, this is typically a nice place to wear it.
Kirkpatrick: But the idea is augmenting memory, basically.
Bell: It’s augmenting memory. That the visuals are then a clue to what was going on then. And now there are people, particularly at Dublin College or Dublin City University that have worn it all the time. And then they’ve worked on the problem of looking at—looking for patterns and summarizing and getting it down to—boiling all the data down to be more and more useful.
Kirkpatrick: But, of course, the software challenge is how do you figure out what to make of all the data that you accumulate. Obviously, this kind of technology will go to video before much longer. Certainly I would prefer to have a video camera that helped me with my quite increasingly failing memory.
Bell: Absolutely. One of the problems here in video and—video, being pictures plus audio, is that the audio—the minute you get audio recording, you then have to get permission.
Kirkpatrick: But not images?
Bell: Images, no.
Kirkpatrick: That’s completely legal.
Bell: Within some degrees.
Kirkpatrick: All you need is lip-reading software, and then you can have the audio.
So the audio is because of the kind of thing that you have to get permission to record a phone conversation with someone.
Bell: Yes, it’s exactly the same.
Kirkpatrick: But Memoto does include audio, right?
Bell: No, they don’t. But they do include GPS. I find the GPS Trails as an interesting—one of these things about your life. That is the fact that I can look at trails much more rapidly, a trail a day, of where was I, sort of this zigzaggy thing that happens. That’s probably one of the more interesting.
Kirkpatrick: Step back a little and explain why you find this so interesting and what it suggests about where computing is going.
Bell: I started the project in 1998 when a friend of mine asked me for my book so they could scan them as part of a thing called the Million Book Project. This was Raj Reddy at Carnegie.
So I said sure, Microsoft has a lot of lawyers. In case we get in any trouble, they can worry about the legal aspects.
So he did that, and then it was about a few minutes after I sent them, I said, gee, I ought to start looking at—I’ve got all this paper that I carried around from time to time. Let’s see what happens if I can take all of that and scan it and then I’ve got all of that. I basically have all of these bit forms that I wanted to scan. And I did that until about 2001 where I was really—the idea was to scan—put everything in my life.
So I basically wrote a paper then. I think it was saving everything.
And at that point, my first epiphany was, gee, I can save it all. It’s easy to do that. I can’t get it back. So it becomes an organizational search problem.
Kirkpatrick: It’s as bad as the fifth of 17 files back in the corner or whatever.
Bell: Yeah. It was piled in my little cyberspace that I had. So we then engaged in MyLifeBits project, in which case we started using SQL to help us organize and categorize and talk about each of these.
And then this device came on the market. We said, let’s do everything. Let’s encode everything we can about life. And so it became MyLifeBits, and that was the project that ran until 2007.
And there was an epiphany where I said, okay—that was about 2005 when I spoke to the SQL database community—and said, oh, what I’ve done is build a transaction processing system for life. So every transactions, every phone call, everything, every mouse click, all of those things, every heartbeat is logged, and then we’re looking—looked at that.
And then we stopped the project in ’07, wrote the book about Total Recall or upload your life. At that point, the epiphany was, oh, it’s all about memory.
What we’ve really done is to create an alternative memory and to create an alternative self. So I think of myself as having a cyber twin that’s all in bits. And that’s the real good memory. And then my—what I’ve got here is a URL and metadata to access that. And that’s where I am now, except—
Kirkpatrick: You mean in here, you’ve got that.
Bell: Yeah.
Kirkpatrick: This is the metadata. You’ve got to somehow digitize that.
Bell: I have URLs. And then I catch the current epiphany of, oh, my God, the data is not just in my hard drive, which we started out 10, 12 years ago, but my data is everywhere. Anything on Facebook, anything on LinkedIn, any statement, anything on Yammer, lots on Yammer, about—that’s all part of MyLifeBits. Things that I may want to—basically from an organizational standpoint, things that I may need to rapidly recall.
Kirkpatrick: Now you’ve also added a huge medical component to this recently. Quickly describe that.
Bell: I’d say that’s my current passion, work, area I’m working in is let’s do all on-body recording of medical data. And I think all of it—I look at it from a computer industry standpoint. I look at everything as it’s the most fun if we’re doing it for ourselves. We’re very self-centered this way, and I’m especially self-centered in terms of working on things that I think I find valuable, and then about 10 years later other people find them valuable.
Kirkpatrick: You pointed to your arm because you’re wearing a measurement device.
Bell: I’ve got—let me slide this one off. This is sort of, I think, the golden device. I’m sure a lot of you have these. This is BodyMedia that has a couple of electrodes here for skin resistivity. It measures heat flux.
Kirkpatrick: This is a commercial product.
Bell: Yes, this is a commercial product. It’s been on the market for 10 years. It’s being used mostly for weight control. Because this thing tells you how much energy you spent. It tells you—gives you a good estimate of how much sleep you get. And so I’ve got those kind of—this is one of the devices. Then I’ve got a body strap here, and then that’s connected to a device here.
Kirkpatrick: Which you keep with you at all times pretty much?
Bell: Yeah, I don’t like carrying this piece of—
Kirkpatrick: He has an iPad, just to promote, considering he works for—
Bell: Wait, this is not really an iPad. No, this is a program that runs heart rate variability.
Kirkpatrick: It just happens it’s on iPad.
Bell: My wife had to buy me one of these.
Kirkpatrick: So that’s why you’re so excited about the Latitude 10 that we’re going to be giving you.
Bell: I want to get rid of this God damned thing.
Kirkpatrick: When he saw the Latitude 10 backstage, he was really getting exciting. I’m not kidding.
Bell: Yeah. But like everything, it’s about the software. This is a $2 program making heart rate variability. So I couldn’t resist the opportunity to take the data here about this to see how my heart rate—how much stress I felt I was under in doing—in being on this, because I’m not—I’m usually fairly laid back in what I do as a researcher. And this is sort of out of my element for now.
Kirkpatrick: Okay. So you’re adding all this data up, and you’re going to have the ability to mine it in various ways.
Bell: Yeah.
Kirkpatrick: Do you think that in a sort of quantified self, contemporary manner you are sort of taking small steps towards the convergence of man and machine that Ray has talked about over the years; or what are you heading towards?
Bell: Well, in this case, I think we’re heading to building a set of tools that really will help everybody, anyone who—particularly anyone with a chronic problem—to be able to diagnose and affect what they do. Already this has had an impact on what I do. I happen to be on my second bypass and second pacemaker and so—
Kirkpatrick: You really are into electronics, aren’t you?
Bell: Yeah. Talk about an in-body. I capture every heartbeat.
Kirkpatrick: So your pacemaker broadcasts out to here?
Bell: No, no, my pacemaker is secure, I hope.
Kirkpatrick: You don’t have a radio on that yet.
Bell: There’s no radio on it. It’s inductively. People talk to it every six months and basically give me the distribution.
Actually, the fact that I recorded my distributions, looked at them, know where my average heartbeat is versus six months ago and the shape of that distribution, I was able to diagnose a problem that said, hey, look, this thing is misadjusted.
Kirkpatrick: What, the—
Bell: The pacemaker.
Kirkpatrick: You diagnosed it?
Bell: Yes. And I read the manual and said this is not something I probably want to be adjusting, because there were lots of words in that manual I didn’t know.
But anyway, it was one of those nice—
Kirkpatrick: We’re heading at a minimum towards a revolution in medical diagnostics and treatment?
Bell: I think so. There’s just so much going on there. In fact, I heard of a device, a group in San Francisco that have made an electrode for EKG capture. Because one of the problems is just getting a good signal, having to wear—I probably won’t try to go through the test today with the strap on, but certainly you can imagine that this causes a problem when you’re in security, when you’re going through a TSA line.
Kirkpatrick: Yeah, I bet it does.
Let’s see if there’s questions. I have plenty more. But who has a question or a comment? Can we get the house lights up to see—right back there. ID yourself, please.
Tugendhat: Good morning, my name is Tom Tugendhat. I’m a British Army officer.
Kirkpatrick: Hey, Tom, I haven’t seen you yet.
Tugendhat: David, you’ve been seeing too many other people.
Kirkpatrick: I’m glad to see you.
Tom is a top defense guy in the UK, actually.
Tugendhat: I have a question about your electronic memory, if I may. One of the ways people deal with traumatic experiences is, of course, that they forget them or reinvent them in their own mind and envision them in a different way. In that way, they cope with life, and we all see ourselves, therefore, as not quite what we are, but as we wish to be. The guy in the mirror is never quite the guy who is really there.
I’m wondering, has anybody done any psychological assessment on what it means to actually remember everything of the last 10 years, rather than what you think you remember of the last 10 years?
Kirkpatrick: What an interesting question.
Bell: Not that I know of. The—there is—there is a really wonderful book about a woman in L.A., whose name I forget. But my computer knows, or can help me remember. But—and I actually may remember it shortly. But she remembers everything. And she’s really quite torn. I mean, the book that she wrote, the fact that she can remember every—
Kirkpatrick: This is just a unique personality trait.
Bell: This is unique. There are few of these people around. You give her a day and time, and she can tell you what TV program or what she was doing and vice versa. You give her a program, and she can tell you what day it was on.
So it was two—it was really a two-way associative memory. Very rare, a rarity. And so her problem is that there’s this turmoil of the stuff that’s always kicking up and around there. And I think these people who have these traumas, you know, where this—it’s the opposite of what we’re trying to do.
We say never delete anything. You know, on your computer, the message here is don’t delete anything on your computer. Someday that piece of information, that e-mail, that letter you wrote, that photograph, even that photograph that was not so good, but—is useful in retrieval is there. Because what you’re creating is an alternative memory. And I call that, that’s sort of the beginning of what I call “light lifelogging,” so you’re not explicitly thinking you’re doing it, but it is just something that happens in the course of not deleting anything that’s ever come through your computer.
Kirkpatrick: And, of course, the storage prices continuing to plummet makes all this possible and will continue to, no matter what we want to store.
Bell: Yeah, the storage and the fact that more and more metadata about whatever is there to make it easier. Particularly one that—ten years ago, we started carrying a GPS logger around so that every photograph—we had these trails and then we had photographs, and then could use the trail to—or the photograph to retrieve—the GPS information to retrieve something about a day or something about, you know, what restaurant was I in. So if you want to know that, you can look at the location and a day and then ultimately navigate back to an American Express bill or whatever to get that.
Kirkpatrick: That sort of thing could be quite convenient.
Okay. Over here.
McManus: Good morning, Mickey from MAYA. Gordon, could you talk a little bit more about afterlife logging?
Bell: One of the things that you can—if you have all of that information, if you’ve been a little bit maybe religious about keeping that, why then what you’ve got is then a legacy that you can have multiple copies of things that you give away to your antecedents, and that way you can have multiple copies of it.
The other thing that we’ve speculated is that you can potentially have an immortality. If you have enough of Ray Kurzweil’s program where you take the existing data and then move forward in time, that one can have conversations, afterlife conversations.
In fact, I’ve done that with a program called MyCyberTwin, and I’ve essentially put in a lot of facts that’s—it’s read a lot of MyLifeBits, and then it’s also taken other facts. And then you can converse with this and get sort of surface kind of knowledge. What was this like, somebody later on can do that. It’s a little harder problem to sort of project me as dead being in the future. That’s a Ray Kurzweil problem.
Kirkpatrick: But the reality is, we’re now living in an age where this kind of thing is becoming somewhat universal, which leads us towards the next panel, in that the Facebook timeline and profile does continue to exist for many, many millions of people after they’ve died, and is considered a form of homage, and that is already exted in the world as a primitive form of what you’re describing.
Bell: No, I think the Facebook timeline is one of those things that provides for an automatic immortality, whether you like it or not.
Kirkpatrick: Just quickly, what do you think of Ray’s ideas of this convergence thing? Do you think we’re going to be able to download consciousness and actually help guide our friends through our memories after we’re dead?
Bell: That to me, I mean, that’s more—wilder than what I think he had projected.
Kirkpatrick: He doesn’t actually say that in any near terms scenario. But his stuff points towards that kind of thing.
Bell: Yeah, the fact that you can get to that much knowledge about consciousness, that’s really extracting quite a lot from what’s going on in the brain. So I think that’s going to take a lot—to me a lot of probing to get at that.
By the way, one of the programs that I’m—one little program I’m running today, in fact—or that I use, it’s a startup called Senti, S-E-N-T-I, and I think I’ve got it set five times a day, it will ask me a question, “how are you feeling?” or “what is going on?’ So it’s trying to get my sentiment.
So in effect, that logging of that—and I’m not exactly sure what the dimensions are. But it’s one of these programs—you know, there’s only—I think we’ve only got a thousand users at this point. But being able to detect sort of an emotional—
Kirkpatrick: So the ultimate mood ring kind of thing.
Bell: It’s the ultimate mood ring where you get that. And then even this program has something of whether I should be doing what or whether I’m in a mood to work or play or so on.
But this particular monitor that is basically called a heart rate variability, which is actually an old measure that—apparently it’s one of these measures. If you can measure heart rate variability, you can prediagnose an impending heart attack. It’s very good for diagnosing food disorders. And I’m using it now.
I wanted—this is a wonderful opportunity to take data. So I said, I’m using it now to see how much stress I’m under. I notice my heart rate is about 50 percent higher than it should be.
Kirkpatrick: Right now? You can see that right on there?
Bell: Right there.
Kirkpatrick: He can see his heart rate right on the iPad.
Bell: Normally I have no stress. My heart rate is 62. My heart rate is 92, and that little stress button is at 1, and this is at 2. And when I was off stage, gee, it got up to 104, which is extraordinary.
Kirkpatrick: Heart rate.
Bell: Yeah. It never gets up there that high. And so now the data doesn’t lie.
Kirkpatrick: Who would have guessed that you were nervous, Gordon.
Bell: I was very nervous.
Kirkpatrick: You don’t seem nervous.
I know we’re really close. Could we quickly get this one last question and a quick response and then we’ve got to wrap.
Burke: Hi, it’s Adrienne Burke from Techonomy. Gordon, you haven’t talked about your genome at all. Have you had it sequenced or are you planning to?
Kirkpatrick: Of course.
Bell: Oh, yeah, I had my genome sequenced. The interesting thing was, yeah, it had a high probability that I was going to have a heart attack, and that was pretty good, because I’d already had two at the time.
The only thing wrong with the sequencing your genome is every—every couple of years, they send you an update. And you—having—I’ve escaped potentially having Alzheimer’s. But then they send you an update, and there are other diseases that are high probability. So you never escape potential diseases.
In the future you say, oh, yeah, it may be only 5 percent. Gee, I just escaped this or that. And so you see this thing coming in, these new diseases coming at you all the time that you may or may not fend off.
Kirkpatrick: You definitely don’t have Alzheimer’s. Thank you so much for coming and sharing your wonderful spirit and ideas with us. Really great to have you here.
Bell: Thank you for inviting me.
Kirkpatrick: Thanks an awful lot.
Click here for a complete video archive of Techonomy 2012.
