Alon Halevy's Blog

Telling Time/Date in Ethiopia (also, a good introductory programming exercise)

2011-02-23T15:18:00.000-08:00

As the regimes in Northern Africa were getting under intense pressure, I flew to Ethiopia to visit coffee farms and experience its coffee culture. I'll write about the coffee aspect of my visit later, but I wanted to share an interesting cultural anecdote first.

Before going to sleep in a traditional Sidama hut, my host announced that we'll be having breakfast at 1:30. I looked puzzled, as I wasn't sure if he's letting me sleep in really late or planning a very early rising. Upon inquiring, I unraveled a fascinating aspect of Ethiopian culture -- the way they tell time and dates.

We'll start with time. The Ethiopian day starts at 6am. So when they say 1am, they mean our 7am (so my breakfast was actually going to be at a reasonable time). Essentially, they are 6 hours off.

Now for dates: the Ethiopian year starts on September 1st (at 6am, of course). They have 13 months. The first 12 months each have 30 days (none of this 31/30 days). The last "month" is 5 or 6 days, depending on whether the rest of the world had a leap year or not. So now, in our February, it's June in Ethiopia (it certainly felt that way).

To make things a bit more complicated, it's now 2003 in Ethiopia. On September 1st, 2011, the year 2004 will begin.

You now have all the information you need to write a date converter into Ethiopian date/time. Seems like a fun programming exercise for an introductory course.

The value of eco-system services for coffee production

2010-09-12T19:29:00.000-07:00

For reasons that will become clearer in a few months, I've become somewhat interested in bio-diversity preservation and eco-system services. Examples of ecosystem services are pollination by bumblebees, decomposition of wastes, and flood mitigation and carbon sequestration by forests.

One of the main problems with ecosystem services is that it is hard to attach to them a monetary value. As a result, decisions to cut down forests, develop lands and interfere with water flows often undervalue these services. Even if there is a value attached to the service, the fact that it is provided by nature makes it harder to tell who should pay to preserve it.

I was recently shown a nice example of where the value of an ecosystem service has been quantified, an no less, in the area of coffee production! In [1], Ricketts et. al show the value of having a forest close to a coffee plantation. They conduct this study in a coffee farm in Costa Rica, and show that within a distance of 1km from the forest, the benefits of forest-based pollinators (i.e., diversity of bees) can increase the production of the farm by 20%. This observation, as they show, can be directly translated to a monetary value. The basic reason that the proximity of the forest is important is that the diversity of bees in the forest enable better cross pollination among plants (whereas, for example, honey bees typically focus on single branches when flowers are dense). Interestingly, the diversity of bees also reduced the number of peaberries produced, which may be slightly more controversial if your goal is to make money off peaberries (which some do).

Thanks to Gretchen Daily for sharing her article with me.

[1] Taylor H. Ricketts, Gretchen C. Daily, Paul R. Ehrlich, and Charles D. Michener. Economic value of tropical forest to coffee production. Proceedings of the National Academy of Sciences, August 24, 2004, Volume 101(34). Pages 12579-12582.

Fun with World Cup Soccer Statistics

2010-06-18T10:27:00.000-07:00

As a teenager I was curious about which minutes in a soccer game are the most likely to have goals scored. I wrote a computer program that stored a database of all the goals scored in the Israeli soccer league for an entire year. I diligently went through all the sports sections of the newspapers and entered all the goals and minutes in which they were scored (feeling very mature that I was able to ignore my strong feelings about some of these goals). I calculated the statistics I was looking for, and the answer was: minute 65 was the most goal-rich minute.

Now, a few decades later, as the 2010 World Cup begins, I find myself asking the same question, or rather, revelling at how easy it is to capture the data, compute the statistics and share them with everyone in the world.

Using Google Fusion Tables, the tool developed by my team at Google, I created the visualization below. We're updating the underlying table as more goals are scored, so you'll always see the latest stats.

But that's not the end of it. Fusion Tables is a tool for data integration. We found some data on fifa.com and joined it with our own table, and then created more interesting visualizations.

This one shows the height of the goal-scoring players. Read into it what you want.

This one shows the distribution of goal scoring among defenders, forwards and midfielders.

And finally, this visualization shows the clubs at which the goal scorers play.

The Checklist Manifesto

2010-02-06T22:02:00.000-08:00

I just finished reading "The Checklist Manifesto" by Atul Gawande, a very interesting book.

Gawande, a surgeon, essentially makes the following point. Given the incredible amount of knowledge we have accumulated in some professions, the complexity of certain tasks could be incredibly overwhelming to professionals (e.g., surgeons, airline pilots). Since in many situations these professionals work under pressure, they often forget some very simple yet important steps that later create unforseen problems (e.g., making sure the antibiotics are applied at a particular time before the incision is made into the patient).

Hence, he argues for simple checklists that teams should go through to ensure that important details are not glossed over. In the airline industry, checklists are used religiously. At every step of the flight, or whenever anything goes wrong, there is a checklist for the flight crew to follow. Gawande's main argument is that this principle should be applied in other professions as well, and in particular, in medicine. He describes his experiences launching such a checklist program with the World Health Organization and the impact that it had on reducing complications following surgery.

There are two main challenges this strategy. First, the checklist needs to be short as to not to completely slow down work. Hence, choosing and phrasing the items on the checklist requires significant thought. The second challenge is putting ego aside. For example, surgeons are used to being the kings of the operating room, and do not lightly take comments from nurses or other staff. Well, pilots have gotten over it, and they're not slackers in the ego department.

Gawande also gives examples from the construction industry and from restaurants, where constructing a high-rise or making sure that everything comes together at the right time on a customer's plate can be rather challenging. One main observation he makes from all of these examples is the importance of communication among the team members, in addition to the checklist. It is crucial for members of the team to communicate well with each other and building communication into the workflow is key. In that way, it's less likely that things fall between the cracks leading to additional problems.

To me the book was interesting because it points out that even if we build a huge body of knowledge in a particular domain, applying this knowledge in practice can be equally challenging.

A Trip to Australia

2010-02-06T21:38:00.000-08:00

I recently returned from a trip to Australia, where I gave a keynote at the Australasian Computer Science Week, the annual gathering of computer scientists from Australia and New Zealand. You can see a journalist's account of what I talked about here.

There is a small but very strong database community in Australia, and I encourage anyone who has a chance to go down under and visit. The strength of the community was apparent when two of the three major annual awards were given for database work. Heng Tao Shen from the University of Queensland received the Chris Wallace Award. This is the top prize given for technical achievements across all fields of computer science (full professors are not eligible for this prize). Heng Tao made his mark spanning the fields of databases and multi-media.

The second award was the Ph.D Thesis Award that went to Michael Cahill who received his Ph.D from the University of Sydney under the guidance of my friend (and excellent cook!) Alan Fekete. Michael and Alan also received the Best Paper Award at SIGMOD 2008 for this work on serializable isolation for snapshot databases.

I was very fortunate to spend time with these winners. Heng Tao was my wonderful host in Brisbane and helped make a long-time dream come true -- sitting on a sunny beach in the middle of January (in Gold Coast). When I went to Sydney, Alan took me to an espresso machine making factory, where I got to see up close how these machines are made!

The coffee in Australia is amazing, and will be the subject of a different post. But if you're going to Australia and need coffee, check out my list of favorite cafes and you'll be happy.

Fusion Tables: The third piece of the puzzle

2009-06-29T06:44:00.000-07:00

When I joined Google in 2005, the goal of my group was to explore the different aspects of structured data and the Web. The first and most burning need was to address the deep web, the collection of databases stored behind forms and invisible to search engines. We developed a completely automated system that has crawled millions of forms in over 50 languages and hundreds of domains. The system surfaces pages from the deep web by guessing good queries that can be posed on the forms, and inserting the resulting HTML pages into the Google index. These pages are shown in the top-10 results for over 1000 queries per second. For all the details, see the VLDB 2008 paper by Madhavan et al.

In a second project, we explored the collection of tables that are already on the surface web. We found over 150 million high-quality tables and developed a search engine for tables (see the VLDB 2008 paper by Cafarella et al. for the details). We also showed how to leverage 2.5 million table schemas that were part of this collection. This collection is now available to the research community.

On June 9th, we launched Fusion Tables, that represents the third piece of the puzzle of structured data and the web. The main goal of Fusion Tables is to make it easier for people to create, manage and share on structured data on the Web. Fusion Tables is a new kind of data management system that focuses on features that enable collaboration. We started with a relatively small set of features, but we’re rapidly expanding them, keeping our users’ requests as our top priority.

You can read the official announcement of Fusion Tables, and a great example of how it is used for data collection in the domain of water. In a nutshell, Fusion Tables enables you to upload tabular data (up to 100MB per table) from spreadsheets and CSV files. You can filter and aggregate the data and visualize it in several ways, such as maps and time lines. The system will try to recognize columns that represent geographical locations and suggest appropriate visualizations.

To collaborate, you can share a table with a select set of collaborators or make it public. One of the reasons to collaborate is to enable fusing data from multiple tables, which is a simple yet powerful form of data integration. If you have a table about water resources in the countries of the world, and I have data about the incidence of malaria in various countries, we can fuse our data on the country column, and see our data side by side. Importantly, we can do this while maintaining complete control of our own data.

Collaboration is not only about integration. Once the data is visible side by side, we may want to discuss it to understand it better or resolve conflicts. With Fusion Tables you can discuss data at multiple levels of granularity: rows, columns and individual cells. Hence, the data and the discussions are deeply integrated (or should I say, fused?)

Given our focus on collaboration, there are a lot of things we do not do (and we're pretty honest about it!). We do not support complex SQL queries or high throughput transactions. Despite our love for query optimization, we’ve implemented very little of it in the current system. We will, of course, add to these capabilities with time, but our real goal here is to explore data management for a broader audience of users and needs.

Please try it out and send us feedback! Our top priority now is to respond to our users' needs.

Coffee: a Competitive Sport

2009-03-09T12:39:00.000-07:00

I went up to Portland, Oregon last week to attend the 2009 United States Barista Championship. No, I was not competing, and unfortunately, not one of the judges either.
You can see all my pictures here.
Portland, by the way, has the largest number of cafes per-capita in the US and has a very active and sophisticated coffee scene (I'm sure you appreciate how hard it is for an ex-Seattle resident to admit this). If you're in town, check out Stumptown Coffee.

The competitors came from all over the country, including the expected share of west-coast baristas and even a guy applying all the charm of a cowboy into his espresso drinks. There were quite a few baristas from Intelligetsia Coffee & Tea, including 4 out of the 6 finalists and the champion, Mike Phillips.

So how do these folks compete? They basically put on a show for 15 minutes, which initially can be quite deceiving because they look incredibly relaxed. The show includes their choice of background music and often some accents in their clothing. In those 15 minutes, they need to prepare espressos, cappuccinos and their "signature drink". They prepare 4 of each, for each of the sensory judges.

All this while, the competitors need to show deep knowledge of their coffee, beginning by explaining their choice of blend, and how each of the flavors comes out in the drink. As they prepare the drinks, they are closely watched by a couple of technical judges, who are watching for every little detail of handling the espresso machine, waste management and timing. Multiple video cameras are following them very closely as they do this, and every now and then the emcee will elicit a cheer from the crowd (let's have it for Mike's first 2 espressos!). If you want to see an example of a wonderful performance, watch the performance of Stephen Morrissey from Ireland when he won the 2008 World Championship.

It was a fascinating crowd from all walks of the coffee industry. There were many spectators in the bleachers, some were huge coffee fans and others who wondered how exactly they got there, but were having a great time anyway. And of course, there was an amazing buzz on the floor around each of the competitors' bars -- after all, everyone in the room was caffeinated...

Finally, the awesomeness of the experience came out most poignantly when I was having a conversation with one of the other attendees and I mentioned to him that I work for Google. He asked: what part of Google do you work for? Food services?

A Report on Healthcare and Information Technology

2009-01-11T12:59:00.000-08:00

I served on a committee of the National Research Council that studied the challenges posed to Computer Science (and computing in general) in the area of healthcare. The report was just released and can be found here.

Needless to say, healthcare is a fascinating area, and progress will require not only collaboration among multiple disciplines (within computer science and others), but also paying attention to the workflow and constraints of the industry itself (doctors' work habits, the way insurance works, or doesn't work, hospitals as businesses, etc).

When reading the report, keep in mind that the goal of the committee was not to analyze what is wrong with the industry right now, but rather articulate the scientific challenges we should be addressing. Gio Wiederhold and Susan Davidson were the other database folks on the committee, which also had experts from other fields of computer science, bio infomratics, and from the medical estblishment.

A Trip to Vietnam

2009-01-03T23:34:00.000-08:00

I just got back from a great trip to Vietnam! (see pictures). Let me first introduce my travel buddy, since he made the trip what it was.

Anhai Doan is a former Ph.D student of mine (he makes sure I emphasize the 'former' part, and adds it himself when I accidentaly forget). Anhai grew up in Vietnam and left after highschool to do his college studies in Hungary, and then went to the U.S for graduate studies, where he ultimately ended up at the University of Washington. He is now an associate professor at the University of Wisconsin, Madison. Anhai is the most well known computer scientist from Vietnam (winning the ACM Distinguished Dissertation Award in 2003 made quite a few waves in Vietnam).

I've been planning to tour Vietnam with Anhai for quite a while now, and a few years ago even decided to tell him about this plan. That turned out to be a great decision. Anhai applied his magic at every step of the way, whether it was whispering the right words in peoples' ears, slipping a well deserved tip at the right time, or shielding me with his body until I was authorized to cross a Vietnamese street on my own. Think of a combination of a (junior, Vietnamese) Godfather-type figure with the potential of some day being a Jewish mother. Since Anhai left right after highschool, he did not see much of the country, and this opportunity gave him the chance to do so.

Anhai does have a few more white hairs than he had before the trip and I take responsibility for that (but I think it's a fair trade for a signature on one's Ph.D dissertation).

We started out in Hanoi -- a city with very strong character with its Old Town full of specialty shops, French Quarter with a very appropriate feel (including the bakeries!), and the promenade around Hoan Kiem Lake at the center. In the middle of Old Town, we found the Green Tangerine Restaurant that was absolutely wonderful (French with Vietnamese influence).

We then went on the mandatory (but very worthwhile) trip to Halong Bay with its many small peaks. We spent a night on a boat there (with a bunch of Australians, the latest invadors to Vietnam). That night was a soccer game (first of two matches) between Vietnam and Thailand, the great rivalry of Southwest Asia. Surprisingly, Vietnam won, which meant the boat crew was ecstatic, and with it being Xmas eve, they started pouring free drinks (accompanied by dried squid from the bay). The second half of the soccer story occurs while we're in Saigon. The next morning, while kayaking in the bay, Anhai and I discovered that kayaking is a team sport, and we have some work to do on that front.

We then flew to Nha Trang, a beach town/resort. It was rainy there, but we still had a good time hanging out in the cafes and even had a schnitzel (I don't think Anhai will ever forgive me for that cultural experience). From there we drove (i.e., sat in the back of a car) to Da Lat, a beautiful city nestled in the mountains, and a resort for Saigonians who need to escape the heat. Also a city with a nice lake in the middle and an amazing variety of flowers.

In Da Lat I was introduced to the Vietnamese 'custom' of serving complimentary tea even when you order coffee. Speaking of coffee, it was a mixed experience. At times, Anhai managed to explain to baristas how to make my macchiatones, and in others we drank Vietnamese coffee.

From Da Lat we drove to Saigon, a much more business-like city than Hanoi and more steeped in history of the American War. The night we got there was the second of the two soccer games, and the result was a draw, meaning that Vietnam won the Suzuki Cup.

Within minutes of the game's end, the streets were flooded with happy Vietnamese. And I mean happy! Literally, you could not move amongst the people and their mopeds. Anhai got me a little Vietnamese flag and we started making our way through the crowds. Being a foreigner and on the tall side by Vietnamese standards, I drew quite a bit of attention. Every time I waved my little flag, I drew cheers and smiles -- almost as if I was the one who scored the winning goal! It was a totally amazing experience!

From Saigon we took a two-day trip to the Mekong Delta, which has now been added to my list of candidate places to retire. Imagine your life when all you need to do for lunch is go to your back yard and fish for a few minutes or pick some fresh fruit from a tree (yes, they have wifi everywhere there too!). It was really fascinating to see how life is arranged when water is such an integral part of your landscape (the delta is actually made of 9 different strands of the Mekong River).

We celebrated New Year's Eve with Anhai's highschool friends in Saigon. That was a great chance to get a glimpse of the life of young professionals in Vietnam, which brought home the point that this country has an amazing future, judging by its people's character and how far they've come in the last 20 years. I'm already thinking of my next trip there!

My Favorite Cafes

2008-12-07T18:59:00.000-08:00

I recently embarked on a coffee-related project. The end product of the project is not completely defined, but it involves visiting cafes all over the world, studying the variations in coffee customs and preparations, and, of course, spreading the word about macchiatones world-wide.

As a first step, I've created a web site that lists my favorite cafes. These are cafes I visited recently, so the list is still short, but I'm hoping to expand it over time. There are multiple criteria for being included in the list, including having good coffee (duh!), having a charming atmosphere and possibly a good location. Basically, it needs to be a place worth spending time at. I'm happy to hear suggestions for additional cafes and I'll find a way to share those with the world.

The Mumbai Attacks

2008-11-29T22:55:00.000-08:00

Pandu wrote a nice post on the Mumbai attacks with a very personal account. Pandu grew up in precisely the area where the attacks took place and knows quite a few people who were affected in various ways.

A Week in Macchiatone-Land

2008-11-28T21:48:00.000-08:00

Oriana and I just came back from a trip to northern Italy where we visited Padova, Trento, Venice and Verona. I'm, of course, not qualified as a writer to appropriately describe the Italian experience, so I'll stick to what I know. Some pictures can be found here.

The reason we went to Italy was to visit the University of Trento, where I was recently asked to serve on the advisory committee of the computer science department. It's quite an impressive place! In about 7 years they built a Ph.D. program from scratch and it now has about 200 students. They've also hired quite a few excellent young faculty. They are able to attract students from all over the world (the courses are in English) and it's a very dynamic place. They also have quite a bit of interaction with local industry. If you're looking for a place to do a Ph.D. (and have great coffee in the meantime), this is a great place.

Which brings me to the second topic. Northern Italy is the mecca for my favorite drink, the macchiatone. Whereas in other places in the world I typically need to explain the concept and get a bad approximation of the real thing, in northern Italy you can walk into any cafe and get a great macchiatone. We even visited the original Caffe Del Doge in Venice (and I'm still recovering from the 5.50 euros for that macchiatone). Later I found out that if you drink your coffee standing, then in most places it will cost a mere 90 euro cents! The official explanation I got for this phenomenon is that in Italy coffee represents a short, intense emotion. This trip has given me quite a bit of material for my coffee project (but I'll explain that elsewhere).

Finally, social networks are really cool when you travel. I was able to collect good recommendations on what to do and where to eat through Facebook (thanks, Andrea!). I enjoyed the comments I got on my status messages and uploaded photos during the trip.

A Tour of South Tel-Aviv

2008-09-23T20:57:00.000-07:00

I recently came back from a trip to Israel to visit my parents. One of the highlights of the trip was the tour that my dad gave me of south Tel-Aviv. For many years, the south part of Tel-Aviv was not considered a great area by any stretch of the imagination, but in the last decade or so it has been significantly beautified and is now one of the more chic parts of the city. You can see all the pictures here.

My dad grew up in that part of town, so it was a great treat to follow him for 4 hours as he was pointing out the various landmarks, and moving about with the familiarity that only a teenager has with his neighborhood (albeit over 60 years later).

The tour started around the corner from the apartment his family rented when they first immigrated from Greece to Tel-Aviv. We then went to what used to be the Alliance School, where my dad went to elementary and middle school, across the yard from the girl's school. In later years, the two schools became meeting points for young activists of the rival underground movements in the pre-state days (the Haganah and the Etzel). Now it is the Suzan Dalal theatre.

We then walked through the Neve Tzedek neighborhood with its charming houses and some Bauhaus-style architecture. We got onto Rotchield Avenue, where the rich people used to live (it's not that much cheaper today either). One of the houses there was where Ben Gurion declared Israeli independence in 1948.

We walked through Herzl St. where my grandfather owned a little store, the big Sephardic synagogue where my two aunts and uncle married and paid a quick visit to my dad's highschool. Finally, we got to the very happening Sheinkin St with its cafes and shops. There we looked for a high-school friend of my dad who had a pharmacy there for 50 years, but apparently retired recently (and by chatting with another pharmacist my dad found out things about his friend that he never knew).

We then sat in a cafe for a bit, and that's where I juxtaposed the present with the past. We got the waitress to take a picture of us with my blackberry and then I showed my dad how I'm instantly sharing it with 200 of my best friends on Facebook. Finally, we walked through the Levinsky Market and spent a bunch of money in a deli before heading home for friday lunch.

A few days later I found myself spending a few hours in Prague. Here are some pictures. Clearly, this was just a teaser and I need to go back.

The Claremont Report on Database Research

2008-08-19T20:48:00.000-07:00

Once every 5 years, a small group of database researchers, practitioners and opinionated professionals get together to assess the state of field. The report written after the workshop represents the consensus on new research areas and describes some of the discussions that took place. The goal of the report is to foster more discussion in the field, so please go ahead and discuss!

You can find the report here.

OpenII: Open Source Information Integration Suite

2008-07-31T21:22:00.000-07:00

Last week I hosted the OpenII kickoff workshop at Google. We had representatives from several companies: IBM, Microsoft, Yahoo, MITRE, Google, one guy who was supposed to represent Oracle but decided to be a professor again, and a couple of professors.

The goal of OpenII, as the name implies, is to create an open-source set of tools for information integration. The tool set will include, among others, wrappers for common data sources, tools for creating matches and mappings between disparate schemas, a tool for searching a collection of schemas, and run-time tools for processing queries over heterogeneous data sets.

The main goal of the effort is to foster innovation in the field of information integration and create tools that are usable for a wide range of applications.

In research, we often innovate on a specific aspect of information integration, but then spend much our time building (and rebuilding) other components that we need in order to validate our contributions. Having a set of open-source tools will enable us to focus on our innovations and perform more meaningful comparisons between our methods.

On the applications side, information integration comes in many flavors, and therefore it is hard for commercial products to serve all the needs. Our goal is to create tools that can be applied in a variety of architectural contexts (e.g., materializing all the data in one repository vs. leaving the data in the sources and accessing it only at query time). In addition, many of the tools (e.g., schema matchers or dedup engines) often need to be extended for the particular domain in hand to fully leverage domain knowledge. Open source tools allow application developers to do exactly that.

You'll be hearing more about this project as we make progress. If you would like to contribute to it, please contact me!

Brain Rules and Computer Science Education

2008-07-31T20:40:00.001-07:00

I just finished reading Brain Rules by John Medina. The last page of the book inspired this post, but let me first tell you what the book is about.

The book examines different aspects of our brain and derives from them some principles and suggestions on how to improve better behaviors. None of these suggested behaviors are surprising at all, but Medina (very entertainingly) explains why they are good for us given the brain's structure and describes some research that validates these claims. For example, chapter 1 tells you that doing aerobic exercise actually improves your mental capacities. Another chapter talks about the need for sleep and another on why stress is bad. Two chapters talk about memory (short term and long term), explaining why repetition of new knowledge can greatly improve its recall. Another chapter explains (finally!) that men and women are different (women are apparently much more complex than men, in case you need another shocker), and another stresses that we never stop learning in life (or at least, we have the capacity to).

At the end of every chapter, Medina asks how we can apply these nuggets of knowledge to develop new methods for education. In the last chapter, he explains why the unique aspect of medical schools make for well-trained doctors as well as curious researchers. The point is that in medical school students are learning the theory of medicine at the same time they are practicing it (in increasing doses as they advance in the program). Hence, they are able to apply their knowledge immediately and, after observing patients, ask novel questions that lead to new research and discoveries. Medina suggests that the same principle can be possibly applied in other disciplines.

I think Computer Science education can really benefit from such a model. I obviously don't have all the details worked out here, but imagine that every Computer Science department (or set of departments) had a software company on the side. As students go through the program, they start getting tasks from that company to build software for it, participate in designs, see how product decisions affect engineering processes, and even see some company politics at work. These companies will be real (they'll need to pay for these services) -- they'll generate real software for real customers.

I even have an initial idea of what these companies can do. Given that these companies are likely to have challenges competing in the market, they need to address a niche of customers who are willing to put up with lousy service, mediocre products and delays in software release cycles. I.e., customers who have nowhere else to go!

These customers are called scientists. Scientists are always complaining that they don't have the right software tools to do their science. Real companies typically don't find scientists to be an appealing set of customers because, well, they don't really want to pay up and they often have very specialized needs. There are huge challenges in creating good software tools for scientists, and university-affiliated companies could be an excellent place to develop this software, while preparing the next generation of computer scientists for the real world.

FOO Camp 2008

2008-07-15T21:27:00.000-07:00

I spent the weekend at FOO Camp 2008, an annual event organized by publisher O'Reilly Media (hence the name, Friends Of O'Reilly). The event brought 275 movers and shakers of the tech industry and related industries, and was an incredible experience. It was as if someone injected into my brain the latest and greatest ideas and thoughts with one joyful syringe, accompanied with a few good glasses of wine. Michael Arrington of TechCrunch captures the spirit of FOO Camp in his blog post (and you can even see me standing and looking busy behind Jimmy Wales, the founder of Wikipedia, in one of his photos).

The conference begins with no set agenda. They put up an empty board with the different time slots and locations of sessions, and as the participants arrive, they fill up the board with sessions. There are about 10 sessions going on in parallel at any given time, most of them looking quite fascinating.

To give you a rough idea, within the span of a few hours, I attended sessions on:

-- aggregating meta-data on the web organized by Esther Dyson (i.e., all the data we create as we use services on the web),
-- the future (or lack thereof) of journalism (organized by several NY Times and SeattlePI reporters),
-- "open education" (tools, policies and politics of),
-- crowd-sourcing vs. curation (i.e., how to balance all the inputs one gets from the bloggers of the world with careful aggregation and analysis of information),
-- how computers can help humanities (e.g., analyzing the Bible, helping archaeologists), organized by Martin Wattenberg, the creator of Many Eyes,
-- educational tools for virtual worlds, and
-- a very well attended session on small things one can do to become happier in life.

There was also a session on "big data", organized by Roger Magoulas, the director of research at O'Reilly. The point I took away from that session is that owners of big data sets are now more confused than ever. They face a much wider array of architectural choices for data management systems than they ever did. These include map-reduce based systems, column stores, real-time warehouses, streaming systems, and various systems built on top of MySQL. Each of these architectures has its advantages and limitations, but it's becoming increasingly harder for application builders to understand the tradeoffs (and it's not like marketing departments are getting rewarded for making the choices clearer). It's no longer the world where you buy your favorite relational database system and you're done (and stuck). I think this situation presents some interesting research challenges for the database community (it's also interesting how some of these architectures get little attention in the community).

The idea of designing the conference program on the spot is very appealing, and I'd like to propose we do a little bit of it in traditional scientific conferences. (There is a concept of birds-of-feather session, but that's usually a grab bag of ideas). We should allot time slots in our conferences where sessions can be organized as the participants come to the conference and stimulate discussions there. That's a much better way of getting up to speed on hot topics and people's current thinking, which is what conferences should be for!

Karina helps the earthquake relief effort in Sichuan Province

2008-06-01T14:26:00.000-07:00

My (almost 7 y/o) daughter, Karina, was a star at a fund-raising event for the victims of the Sichuan Province earthquake. I'll give a bit of background, and then I'll let the quotes from the press speak for themselves. This event was originally supposed to be a victory party for the newly elected president of Taiwan, and hence highly star-studded.

A few hours before the event (a crowd of 5000+), Karina got a call asking if she would come on stage for a few minutes during the fund-raiser. Nobody told her what was expected of her, but she was happy to participate. She got on stage and was asked a few questions, at the end of which she offered the emcee to share a poem with the crowd. The poem recital apparently moved the crowd greatly and opened up their check books. Her impromptu performance was covered in the Chinese press following the event.

For the readers who do not know Chinese (e.g., me), here are a few snippets from the articles (kindly provided by Oriana). Also note that most of the Chinese press uses her Chinese name -- CunZhong, but I assure you she could have done it perfect Hebrew too.

Notable quotes from the Sing Tao site: (1) "This first-grader was born and raised in the US. Her mother is a Chinese from Beijing, her father Jewish. Standing in the center of the stage, wearing her hairs twisted high into two traditional buns and holding her piggybank, little CunZhong delivered an original poem, entitled "Home", in impeccable native Mandarin Chinese." (2) "Thunderous applause followed her performance." (3) Karina told the reporter: "Television images of the earthquake victims really scared and worried me. I memorized this poem after reading it three times. I want to share it with all the children in the disaster zone, so that they will not be afraid!"

This site carries the original news release with a photo. This site also contains several photos from the concert, including one where the audience responded to the emcee's tribute to "the mom of this courageous little girl." The notable quote from the site is: "Li CunZhong's outstanding and emotion-filled performance deeply touched the hearts of the entire audience, and brought the outpouring of donations to a crescendo."

And finally, here's the YouTube video with the post-interview with sina.com, titled "Karina, the six and a half year old who touched many hearts with her live recitation of the poem". If you're Chinese, the management requests that you go here to see the video and read more background.

The Macchiatone

2008-06-01T13:34:00.000-07:00

Do you often feel like your cappuccino has too much milk? And then the macchiato is a bit too dainty? A conundrum I'm sure many people face on a daily basis.

This morning I went for my usual sunday coffee at Cafe Del Doge in Palo Alto, and I had the chutzpah to point out that their cappuccinos have too much milk. After a short discussion, the barista pointed out to me that I should try the macchiatone. It's basically a cappuccino, but with less milk. I've been making macchiatones for several years now and I didn't know it!

Something tells me that the Starbucks barista course does not cover this material, but feel free to spread the word (note: a macchiatone is NOT a dry cappuccino; very different concepts!)

The Remote Agent

2008-06-01T09:01:00.000-07:00

I just returned from the 5-year Database Self-assessment workshop (will post something about this soon), where some of the discussion naturally focused on identifying high-impact ambitious projects for the community.

When I returned, I found Pandu's blog post, describing the Remote Agent Project at NASA that he was involved in about 10 years ago. Now that was a very inspiring project that received a lot of attention in the AI community at the time (including a Best Paper Award in AAAI 1997), and really demonstrates the amazing things that can happen when an incredible group of people get inspired.

I wonder if the database community can come up with something as inspiring.

Israel at 60!

2008-05-07T22:17:00.000-07:00

Last year I blogged about Israel's Memorial Day. This year I'll focus on the second (and happier) part of this two-day national event -- Independence Day, that immediately follows Memorial Day. And quite a day it is this time -- Israel is celebrating its 60th birthday!

I will not go into all of Israel's achievements in its first 60 years, but you can find a few listed here. Instead, I picked up the phone (ok, Skype) and had a chat with a member of Israel's founding generation/team -- my dad. Here are a few of his comments.

First, I wanted to know what he felt on the day that Ben Gurion declared independence?

Turns out he only heard about it two days later. He was in the middle of a battle in Jerusalem, and two days after the declaration, he was injured by a bullet and was taken to a medic for treatment. The woman in charge of the medic (who later became Israel's first lady) came by and told them it had happened. Not a big surprise -- they were actually expecting it to happen sooner or later.

Second, I wanted to know what they were thinking those days.

Not much. They wanted to stay alive and make it through another day. Not surprising -- they were constantly being shot at. Interestingly, they believed that once the war is over, the conflict will be done and there will be peace. They certainly did not anticipate having to fight wars 60 years later.

Then I asked my dad whether he's happy with the result 60 years later.

There were two parts to his response. He claimed that he and his cohort did not imagine Israel would have so many achievements and grow to be the strong country that it is. Their expectations were exceeded by far. On the other hand, he claims his 1948 cohort were a bunch of 20 year/old idealists. They thought their idealism would pervade all walks of life in the country they created. But today there is too much "business/politics as usual" in the country. All aspects of human nature are represented, and perhaps that's inevitable.

Finally, I turned to my mom and asked her what was she thinking when she emigrated from the US to a 7 year-old state (she emigrated after marrying my dad in 1955). She answered, and I quote: "I've never met anyone so passionate about his country (like my dad). It was a great adventure to come here. So what if they didn't have toilet paper".

Hair-Care Records

2008-05-01T21:02:00.000-07:00

There has been much talk recently about electronic health-care records. Imagine that whenever you go to a health-care provider, your health-care record is available to them, thereby enabling them to give you better care. There is a lot of controversy about the implementation and the privacy concerns and policies surrounding such records, but ultimately this will happen.

I'd like to propose a simpler and much less controversial idea. Last time I went to get a haircut, I realized my hair dresser had no idea about my hair history. She didn't know when I had it cut last, how short and what style I asked for then, and whether I liked the result or not. And being a busy guy, I couldn't recall all the details myself either.

So why not create electronic hair-care records? Every time you go for a haircut, you get before and after pictures, with a time stamp, and a few comments attached from the hair dresser. Now you can take this record with you wherever you like and next time you come in the discussion can focus on more important issues.

There are more benefits. You can highlight a particularly good haircut and always ask the provider to mimic that. If your hair is especially challenging (in a good way), you can auction your haircut to a hair dresser who wants to boost their resume. If you're anywhere in the world, say, Patagonia, and you feel the urge for a haircut, simply whip up your record and no words are needed. After all, hair is a universal language.

Technologically, building these records is simple. We just need someone with the right entrepreneurial spirit.

Crawling the Deep Web

2008-04-14T21:36:00.000-07:00

Our work on crawling the Deep Web has received some attention over the last few days. It started with a post on Google's Webmaster blog. Judging by the number of in-links to the blog (see the bottom the page) and the several news articles that picked it up, there were quite a few reactions on the blogosphere and beyond.

Matt Cutts, Google's main interface to web masters gives a nice explanation of why this work is useful to site owners. Anand Rajaraman details some of the history behind the technology that led to this work.

In summary, a nice example of research on data management having impact on the Web.

Bar-coding in Costa Rica

2008-04-02T21:04:00.001-07:00

I spent last weekend in the Area de Conservacion Guanacaste (ACG) in Costa Rica with a few of my colleagues. We were hosted by Dan Janzen and Winnie Hallwachs (his wife), an incredibly inspiring pair of biologists. Among many other awards, Dan is also a recipient of the Kyoto Prize in 1987. For the past 30 years, Dan and Winnie have spent half of every year in Costa Rica, creating the ACG, while spending the other half professing at the University of Pennsylvania. I'll have to skip the details of how we got there, but do ask me in person when you see me (and if you need to juggle my memory, use the phrase "party in the sky").

You can find the pictures from the trip here.

So you're probably wondering what was a guy like me, with questionable credentials in Biology, is doing in such a biologically intense area?

Imagine that every living species and plant had a barcode, just like products in a supermarket. Furthermore, imagine that you had a device, the size of a cell phone, such that when you found a specimen in the forest, you can put the specimen into the device and it would tell you all the known information about it. In addition to being a useful device to take on hikes, such a device can have major impact on agriculture and controlling the spread of disease.

The International Bar-Code of Life Project (iBol) is trying to do exactly that, based on genomic techniques. Specifically, it turns out that with over 98% accuracy, the CO1 gene uniquely determines the species. In contrast, the traditional approach to determining species is based on morphological features. By sequencing the CO1, Janzen and many others have been able to uncover several mysteries, showing that species that look very similar are actually different, and vice versa. Janzen runs the biggest specimen collection operation (Costa Rica happens to have a huge number of different species, hence Janzen's conservation goal). Currently he sends them to the University of Guelph in Canada for sequencing (in a lab run by Paul Hebert who was also there), but they envision that in a decade, we'll be able to build the small device.

We spent the weekend in numerous and intense discussions on Biology, walking through the forest seeing it first hand, and actually participating in the process of collecting specimens and preparing them to be sent for sequencing.

In the discussions we tried to understand the challenges involved in this project (including arguments by its critics). It actually turns out that determining species can often be very subjective, for two reasons. First, the determination typically needs to be done with only partial information about the set of specimens available and unless you can find other evidence, morphology is typically the deciding factor. Second, and somewhat more surprising to me, not all biologists completely agree on what the concept of species even means. The most accepted definition is based on the ability to mate and create viable offsprings, but there are other opinions as well (e.g., it's the morphology stupid). In fact, when it's not even clear (to me, at least) that classification into species is as important as it's traditionally been considered, since many of the questions we're asking about animals or plants depend on other genetic and environmental traits.

And yes, there are huge data management challenges here. Many scientists are collecting data and each putting it into their own format. They would like to share their data but also maintain control of their own. They'd like to publish the data on the web and make it accessible to the masses. They need to manage uncertainty and provenance. Ironically, one of the closest systems I know that is considering some of these issues is Orchestra, built by Zack Ives at the... University of Pennsylvania (i.e., a few buildings away from Janzen's office).

Then there was the flight back, but I can't talk about that either. Overall, an incredible experience! Many thanks to Dan and Winnie (and their crew) for hosting us and sharing their incredible knowledge and passion!

Two Books by Geraldine Brooks

2008-03-16T20:50:00.000-07:00

I just finished reading two books by Geraldine Brooks and highly recommend them. Both books are fiction inspired by true historical events. In both cases, Brooks manages to vividly recreate the periods in which the plot is taking place and bring them back to life. The research that goes into her books is really impressive.

The first book is Year of Wonders. It is based on the story of the little village of Eyam in Derbyshire, England in 1666. That was the year plague swept through the village devastating it. The story tells of how the plague entered the village and its effects on its inhabitants through the story of Anna Frith, a housemade at the village's rectory.

The second book, People of the Book, was published just this year. It is based on the story of the Haggadah of Sarajevo. The Haggadah is the story of the exodus of the Israelites from Egypt, and is read on the eve of Passover (with a lot of food and a minimum of 4 glasses of wine weaved in). The history of this Haggadah goes back to 15th century Spain and has an amazing story of survival through Venice, Vienna and Sarajevo (at least). One of the most interesting aspects of its story is that the acts (often of heroism) to save the Haggadah were typically carried out by non-Jews -- Muslims or Christians, who appreciated the value of the book. The book portrays vividly several periods in history some of which had Christians, Muslims and Jews were living in peace together (Spain, before the inquisition). It's really a great read (regardless of one's religion). And yes, there was even a Halevy involved in this book's history!

A "Web Moment"

2008-01-28T20:57:00.000-08:00

I'm sure each and every one of you has had at least one "web moment", where the power of the web simply jumped out at you. It may have been after a web-search yielded an amazing result, or realizing that you're driving to a dinner meeting at http://... using directions you got from online maps, and checking traffic conditions on the web from your car. (I do realize, however, that there is an entire generation out there who has no idea what the heck I'm talking about, and equates the pre-web world roughly with the 19th century).

I (or rather, my dad) had such a moment yesterday, when he found a document on the web, signed by his father in 1956, that he had no idea existed.

Yad Vashem, the Holocaust Museum in Jerusalem, collected a database of people who were killed in the holocaust. They asked anyone who knew holocaust victims to report their details and contribute them to this database. I found the web site for searching that database (thanks to Yair Kurzion), and told my dad about it.

His task was not easy. Searching for a Levy in database of Jews is like searching for Smith in a phone book of a big American city, and even restricting the search with the first name and the city of origin did not help a lot. But magically, he pulled up the first result and found a document signed by my grandfather (who passed away 40 years ago). My grandfather had reported the death of his brother and sister, who were both deported from Thessaloniki, Greece to the gas chambers in Poland in 1942, along with the vast majority of the vibrant Jewish community of that city. Fortunately, my grandfather, who was a Zionist at heart, left Thessaloniki in 1933 for Tel-Aviv to later be part of the creation of the State of Israel.

My Dad is 80!

2007-12-28T22:14:00.000-08:00

My dad turned 80 this month, and we celebrated the event with a workshop and reception in his honor at the Weizmann Institute of Science, in Rehovot, Israel. The full set of pictures from the event can be seen here.

My dad is a professor of Chemistry at the Weizmann Institute. After fighting in the Israeli War of Independence, he was finally able to focus on his studies. He completed his Ph.D in a little less than 2 years(!!) in 1955 at Syracuse University in New York (fortunately, because that's where he met my mom). When he's asked how he did that, he simply shows the picture below.

He still doesn't understand why it took me an entire 5 years to do my Ph.D, and worse, in a field that uses the term 'science' in a questionable fashion. (When I got promoted to full professor he finally figured I might be doing something right).

My dad's main claim to fame is a 1-page article he wrote during his post-doc. The following makes the point better than I can - it's a quote from Krzysztof Matyjaszewski (a CMU professor) and Axel Muller (professor at the U. of Beyreuth, Germany) in their foreword to the December 2006 special issue of the Journal of Progress in Polymer Science on "50 Years of Living Polymerization":

On June 5, 1956, Michael Szwarc, together with Moshe Levy and Ralph Milkovich pubished an article entitled "Polymerization initiated by electron transfer to monomer - A new method for formation of block copolymers", J Am Chem Soc (1956), 2656.

In this article the term "living polymer" appeared for the first time. It caused a revolution in polymer science.

Michael Szwarc (who was also my dad's Ph.D adviser) received the Kyoto Prize for this work in 1991.

He has worked in many areas over the years, but since the mid-80's my dad has been one of the pioneers in solar energy research, studying methods for chemical storage of solar energy so it can be used any time and transported to less sunny locations. In fact, he published two papers on using solar energy for chemical transformations this year! As a befitting token or recognition, he received an awesome Google solar t-shirt...

And he definitely needs the t-shirt. He still gets up every morning at 6am to either play tennis, or go for a run & workout, which includes running up 15 flights of stairs in the solar tower at the institute!

It was a great event, and a wonderful chance to see many of my dad's colleagues throughout his career, some of whom I had not seen since I was a kid. It was also the first full gathering of all the family's grandchildren.

Dataspaces for Veterans

2007-11-11T17:10:00.000-08:00

In the U.S. we are marking Veteran's Day tomorrow. There are many ways in which we should be thanking our veterans and making their lives better. I'd like to report a rather unique way.

I recently had the opportunity to visit the Veteran's Administration Hospital in Washington DC and learn first-hand about their patient-record system. I was pleased to see the principles of dataspaces in action, clearly enabling better healthcare services.

The VA provides services to veterans of the American military and has around 150 hospitals, 800 clinics and 200 nursing centers scattered around the country. To support these services, the VA maintains electronic records for all their patients, a system that has won them many accolades in recent years. The system stores the patients' prescriptions, doctor visits, lab tests and other data about each patient. As their patients often move around and receive treatment in various locations, when a doctor views the data about a patient, it needs to be integrated from multiple VA locations. Each of these locations is running their own system. In addition, data about their patients may reside in systems of the Department of Defense (and their healthcare providers) and various drugstore chains.

Clearly, this is an incredible data integration problem. Today they are aware of at least 130 different "implementations" of their electronic record system, i.e., different schemas. Also, given the different local needs of hospitals and clinics, imposing a single schema on all the VA centers would not work. Using a data integration solution at this scale and in such a dynamic environment would be extremely difficult.

Instead, what the VA did is standardize on a very small subset of patients' attributes, namely attributes describing patients' vital signs. Outside of this set of attributes, hospitals are free to develop their own local data organizations. However, the system lets the healthcare providers see all the data even if it's not completely integrated. So for example, if a doctor wants to see what happened to a patient while they were at a remote location, then the remote data may appear as plain text, and therefore the doctor would have to work a little harder to digest it, and won't be able to pose the queries she could pose on the local data. But being able to see the data in some form is infinitely better than not seeing it at all, and the doctors are extremely happy with the system's capabilities.

The VA also demonstrated two examples of the pay-as-you-go principle that is at the foundation of dataspaces. The first was the fact that they decided that vital signs are critical, so their data sources are aligned on the attributes relating to those (effectively, creating semantic mappings involving the attributes of vital signs), and they plan to continue agreeing on terminology as they see fit. Second, they had a culture that allowed for local innovation, class-3 applications, that represented needs at the local level. When these needs were perceived to be important throughout the organization, they promoted them to class-1 applications, and required all their systems to support them.

Just to make it clear, when I walked in the door they did not greet me and say: "Pleased to see you Dr. Halevy; we'd love to show you our dataspace system". What I'm describing is a post-rationalization of a system that was developed over more than a decade. I believe that their loose integration was the key to their success.

A Murder Mystery with a Twist

2007-10-24T20:32:00.000-07:00

I just finished reading dot.dead, a Silicon Valley murder mystery by Keith Raffel. Yes, a murder happening in Palo Alto at the home of Ian Michaels, a high-tech executive; searching for clues on Stanford campus and running the dish to think deep thoughts and unravel the mystery.

Will Ian Michaels be indicted and spend the rest of his life in jail? Or perhaps he will be promoted to that COO job he's been eying for a while, or even leave his company and start his own? And in the process, how many eligible (or non-eligible) women will try to seduce him?

Read the book and find out. Not bad for an author who used to be a high-tech guy himself.

A Trip to the Amazon

2007-10-20T20:48:00.000-07:00

You can’t go to Brazil without going to the Amazon. So after giving an invited talk at SBBD 2007 in Joao Pessoa, I headed to Manaus, the capital of the Amazon region (see all photos here). It took 3 flights and some time-zone slalom (while I was flying west all the time, I had to turn my clock forward at one point, and backward at another). I landed in Manaus at 9pm, in the dark. I gentleman named Eduardo was waiting at the airport with my name on sign held up high. We got into a van that started driving towards the docks. It seemed that we were going through some pretty heavy forest in country roads into the darkness of the night (though I was reading a murder mystery during the flight, I tried to keep a positive attitude here).

We arrived at the docks and I was handed off to an unnamed man with a motorboat. I was told that it would take 20 minutes to get to the EcoPark. As I was sitting there, sailing in pitch dark (with only the moon in a very southern-hemisphere position to provide a bit of light), the sensation of adventure started sinking in. The boater turned on his flashlight every now and then to see where we are, and surprisingly, 20 minutes later, we arrived at a lodge, and I met Antonio, who would be my guide for the stay. After a welcoming drink and a short hike in the forest to my cabin, I plugged in my cell phone and went to sleep.

In the morning, after a yummy Brazilian breakfast, I went on a forest tour with Antonio and John, and American fellow who is 10 years into his retirement (he’s 48 now) and whose travel plans for the next couple of months made even my head spin. In the forest, we got to see an original cinnamon tree, a tree from which they produce “aborto”, which, as the name implies, is used to abort pregnancies (and is also a useful post-hangover medication as well). I got to be tarzan for a photo and see Antonio make gunpowder from scratch. Really. As we were finishing our forest walk it started raining (ah, get it? RAINforest, in this case pourForest would be more appropriate), and we were thankful it didn’t start earlier.

After lunch, I was introduced to cashew trees (yes, apparently they grow on trees, not at Costco). The barman took the cashew fruit and made a nice drink out of it. Later we went to an area with a bunch of monkeys playing about, including one with a red head and one that was called cappuccino monkey (if you’d see it, you would understand why). We were taken to see a few folks from the local Indian tribes (and ended up in funny costumes, doing their dance), and then spent an hour fishing on the river. I even managed to catch a catfish that I threw back into the water once the photo-op was over. Sitting on the fishing boat was incredibly tranquil, with sounds of the toucans flying about (and the news of the latest Google earnings report coming in on my cell).

I had dinner with a retired Swedish couple in their late 50’s. The husband ran an international company providing interior design services for cruise ships. He admitted that when a good friend of his came to him many years ago with sketches describing his idea for an ice hotel, he told his friend that he was crazy. Fortunately, the friend ignored his advice and did it anyway. (Oriana and I got engaged in that ice hotel in 2000).

The night activity involved a canoe trip on the river, looking for caimans and listening to the night sounds. We saw only the eyes of the caimans from afar, caught a couple of turtles, and heard many frogs. When we returned to the lodge, Antonio thought he heard lightening, but immediately corrected himself – they were only fireworks. Why fireworks, I asked? Because there was a soccer game in Rio between two teams from Rio, and the people in Manaus were very happy with the result. And it wasn’t even a terribly important game. But that’s Brazil for you! Celebrating a soccer victory of a team from a city thousands of miles away in a relatively unimportant game, is still good enough reason for fireworks.

The next morning, before going to the airport, we managed to squeeze in a speedboat trip to the meeting of the waters – the place where the Rio Negro and Rio Solimoes come together to create the Amazon River (that then flows to the Atlantic ocean). It was really fascinating to see the two distinct waters (they’re different in color, temperature and PH).

Five hours later I was in Miami, and six hour after that in the bay area. What a transformation!

SBBD 2007

2007-10-20T20:04:00.001-07:00

I just returned from a wonderful trip to Brazil, where I and another funny looking person, gave keynotes at the Brazilian Database Conference, SBBD 2007. The conference is held jointly with the Brazilian Software Engineering Conference, and had over 600 attendees. You can see all my photos here.

The conference was held in Joao Pessoa (read: John person; apparently he was a poet), which is in the eastern-most point in South (or North) America, the closest place to start a swim to Africa. The venue was a tropical beach resort. I'm not sure when the residents of Joao Pessoa sleep. They seemed to be dancing all night, and at 5am the road along the beach was closed till 8am so people can exercise peacefully. I guess they need all the dance and exercise to burn off the calories from their great food.

Many thanks to Altigran Silva for inviting me and being a wonderful host, to Juliana Freire for all her incredible help and company, and to all my Brazilian friends for their great welcome (yes, my Orkut network has grown exponentially as a result of the trip!)

A Geography Quiz

2007-10-15T12:08:00.000-07:00

China and Brazil. Both very big countries. One of them has a single time zone throughout and the other currently has 7 time zones. Which is which?

You were wrong.

The answer is: Brazil 7, China 1. (Sort of what you'd expect if they played soccer against each other).

I was in China earlier this year and went as far west as Tibet (parts of which are about as far from Beijing/Shanghai as San Francisco is from New York). Still, a single timezone throughout.

In Brazil the situation is much more complicated, as the country is challenged both in longitude and latitude. Ordinarily, Brazil spans 4 time zones. However, now they just moved to daylight saving's time.

If you live close to the equator, then the concept of seasons is rather abstract, so DST makes no sense and the northerners ignore it. As a result, the northern part of the country spans 4 time zones on standard time, and the southern part spans 3 time zones on daylight savings time (I realize this means there is an overlap between the timezones so if you're being strict there are only 4 or 5). But if you're flying around Brazil, as I'm doing now, there are 7 zones for you to grasp with.

I flew in to Rio on Saturday, a few hours before they moved to DST. On sunday morning, having successfully woken myself an hour early, I flew to Joao Pessoa, where they're not on DST. Needless to say, things are messed up a bit, but in Brazil you just drink a little more and relax.

The Oxford Murders

2007-09-30T22:00:00.000-07:00

Want to read a murder mystery with a mathematical twist? "The Oxford Murders", by Guillermo Martinez is a nice one (apparently, the movie is coming out next year).

A series of murders is happening in Oxford, tangentially involving people in the math department there. Supposedly, there is a mathematical series underlying the murders, and a couple of mathematicians are trying to figure it out before the series goes too far. A very nice and rather quick read.

VLDB 2007 Trip

2007-09-30T21:28:00.000-07:00

I just came back from VLDB 2007 in Vienna and from giving a 2-day data integration course at the University of Aalborg, Denmark.

Traveling in Europe is always fun. I find it much more relaxing to assume (with some loss of accuracy) that the Euro and the American dollar are about equal in value. It seems more affordable this way. I've never been to Vienna -- a very nice place to visit. After spending 5 days there, I had 5 schnitzels (a favorite childhood dish of mine) and a large but finite number of Viennese cakes & tortes. Fortunately, I did have the opportunity to go for a couple of jogs while there, so am still able to fit into my clothes.

In Aalborg I gave a course on data integration. This was the first time I gave lectures based on the first few chapters of the book I'm writing with Zack and AnHai. The energetic students in Aalborg helped me debug the slides and the presentation, and overall it was a great experience. Though I knew this before, the database group in Aalborg is a very strong one and doing some exciting work (I was initially surprised to see that all the rooms in the department were labeled DatalogI, but then I was told this means computer science in Danish). My host, Christian Jensen, was very kind, and after making sure I got my exercise, took me up the coast to some very charming towns.

VLDB was very interesting. Yet again, I was pleased to see a lot of work going on in the area of data integration, uncertain data, web, etc. I attended two excellent tutorials: Adaptive Query Processing by Zack Ives and Amol Deshpande, and on Probabilistic Graphical Models and their application to data management, by Sunita Sarawagi and ... Amol Deshpande.

The high point of the conference was no doubt the video shown during the 10-year best paper award talk by Surajit Chaudhuri and Vivek Narasayya. The extremely hilarious video showed Surajit giving a demo of Auto-Admin along side Bill Gates during Gates' keynote at SIGMOD 1998. To put it nicely, the video showed Surajit's ability to mask certain unexpected mishaps during the demo and make it all appear to go extremely smoothly.

A few words on DBclips. Everyone I speak to says it's an excellent idea. However, so far, very few people created them for their papers. Since Luna posted the DBclip on our paper less than 2 weeks ago, it's already received over 180 views. Need I say more?

There are simply too many interesting talks and other events going on in parallel during most conferences. There is no way anyone can make it to all the talks of interest to them. Sadly, even people with the best intentions will not have time to diligently go through the proceedings and read all the papers either. A DBclip is an excellent way to reach a wider audience of people. While it takes some effort, it's well worth it. In fact, during my course in Aalborg I showed our DBclip instead of lecturing on data integration with uncertainty. I imagine that these clips will be a useful teaching resource in many graduate courses. Fortunately, it's not too late. You can create your DBclip after the conference (in fact, there are advantages to doing it now).

Web 2.0 Panel

2007-09-27T00:57:00.000-07:00

This post is an experiment. We're holding a panel at VLDB 2007 on Web 2.0 and Databases. What are the opportunities for database research?

Our panelists are Sihem Amer-Yahia, Gerhard Weikum, a Donald Kossmann lookalike, Volker Markl, Anhai Doan and myself.

Given that the topic is Web 2.0, we thought the audience should post comments and opinions throughout the panel. Feel free to say anything. We'll monitor the blog during the panel and highlight your comments.

Comment away!

DBclips

2007-09-17T22:25:00.001-07:00

VLDB 2007 has announced a new program called DBclips. You can now upload a 5-minute video of your VLDB paper. (This idea may sound familiar to the readers of this blog).

You can listen to my first DBClip, created for the paper I wrote with Luna Dong and Cong Yu, on Data Integration with Uncertainty (also an idea that's been mentioned in this blog).

I can't wait to hear others' contributions!

The Lives of Others

2007-09-11T21:25:00.000-07:00

The Lives of Others recently came out on DVD (i.e., now available to parents of young children). We just saw it -- highly recommended!

I will not even try to write a review of this movie. Just google for reviews.

Anant on the Database Blog

2007-09-11T21:20:00.000-07:00

Another fine post by Anant.

Sergey's Story, in Chinese

2007-08-13T18:44:00.000-07:00

A few months ago, Moment Magazine published an article written by Mark Malseed telling the story of Sergey Brin (Google co-founder).

My wife Oriana, in addition to being a lawyer, is quite a skilled English-Chinese translator (that's an understatement, believe me). She translated the article (under the obvious assumption that anything that is interesting to Jews would be interesting to the Chinese). And in fact, her translation was recently published in the Sunday Weekly Magazine editions of the SingTao Daily, the most widely circulated Chinese language newspaper in North America.

You can get the full translation here.

As with any Chinese-related content on this blog, the usual disclaimer applies: it's all Chinese to me.

Two Zodiacs Away

2007-08-08T21:24:00.001-07:00

I feel like I should have something to blog about on my birthday. Next year, it won't be a problem -- the Chinese have scheduled the opening ceremony of the olympics for that day.

Generally, I'm a guy who feels pretty comfortable with his age. That's an important survival skill when you work for Google. Every now and then, however, I realize that even though I'm not feeling any older, time is passing.

Today the poignant moment was when I was standing and talking with my group members and interns (all very nice, organized a nice birthday celebration. Special thanks to Bijun who ordered a chocolate babka from Zabars in NYC!). I was chatting with my youngest intern (a graduate student) who mentioned she was also born on the year of the rabbit, like me.

Then there was a pause. It took us both a mere 5 seconds to realize that she's *2* zodiacs away from me. Not one. There is an entire unrepresented rabbit in between us.

While it should be said that she is the youngest graduate student I ever worked with, I still found it startling that I'm working with a rabbit two zodiacs away. I'm sure I'll get over it.

Notes from Ed Tufte

2007-08-01T20:35:00.000-07:00

Sorry for being silent in the last while. Blame it on the book I'm writing (on data integration). I had to finish a chapter before I could write anything else.

After several years of getting the brochures in the mail, and watching Tufte's book on envisioning information sit happily on my shelf, I decided to go for his 1-day course in SF and learn a thing or two about effective visualization.

If you want the full experience, I recommend sitting next to someone who worked on Microsoft Powerpoint at some point in their career. I did that, and it added quite a bit to the entertainment factor of the course. Tufte's powerpoint rant starts about 5 minutes into the course and he makes his last jab in his closing remarks. But more on that in a bit.

The course was interesting, even if it mostly gets you thinking about issues relating to effective visualization. I jotted down a few notes that I'm repeating here mostly so I don't forget next time I'm preparing a presentation. Most of them are obvious, but that doesn't mean they're not often forgotten.

more detail in your presentation increases your credibility
more detail does not necessarily imply clutter (if done right)

annotate anything you can in a visualization. For example, annotate links (otherwise you're implying that they all mean the same)
don't try to be too fancy. Focus on the content not on the design. For example, tables are a very effective, yet simple presentation. Order the rows in the table according to some performance measure you're trying to emphasize.

don't focus on being original in your visualizations, focus on getting it right (don't innovate, steal).
get users out of the decoding business (i.e., remove legends where possible)

The deeper point made in the course is that principles underlying creating effective visualizations mirror the principles that underly thinking processes. Hence, for example:

Make and show comparisons between different aspects of the data
Make sure causality of effects is emphasized in the presentation
Build credibility -- make sure you show all the data rather than just cherry-picking what's convenient for you.
Enable the audience to drill down and see more data.
Integrate evidence from multiple sources (aha, a plug for data integration!)
Always give the source of your data (yes, lineage, folks!)

Following the principle that good presentation should support critical audience thinkers, Tufte also points out what audience members should keep in mind as they listen to a presentation (surprisingly, reading your email on the blackberry is not one of them)

What is their story?
Can you believe them? Do they have a any conflicts of interest affecting their perspective? What's their track record? What's their reason for bias?
What precisely does their argument apply to? What are its limits?
What do I really need to know when I leave the room?

Ok, back to the powerpoint issue. I actually found myself a bit confused throughout the main point of his powerpoint rant, but I think I get it now. His basic point is that powerpoint forces you into a very low resolution presentation mode. He argues that people can read 3 times faster than you can talk. In addition, powerpoint encourages you to leave quite a bit of detail out and summarize everything in bullets. The human brain can take in much more than what you can convey with a powerpoint presentation. Hence, you're not really using your time with your audience very effectively, since there are better methods of conveying information that make much better use of the audience's mental capabilities. For example, he argues that you should come into a meeting with a 3-4 text summary of your points, have your audience read it, and then have a discussion and answer questions.

The latter suggestion makes it pretty clear when his methods are effective and when not. For example, it's a non starter for large audiences (e.g., conference presentations). On the other hand, there are cases where we do this by default (e.g., hiring meetings don't typically involve slide presentations). So, don't dump powerpoint just yet.

Interestingly, Tufte was not following his own advice very carefully during the day. I felt that the principles he espoused could have been communicated more efficiently (but then, perhaps he assumed that some of the audience also had blackberries to attend to).

SIGMOD 2007

2007-06-25T20:36:00.000-07:00

I just returned from SIGMOD 2007 in Beijing. This was only the second time SIGMOD has gone out of North America, and it was quite a success. The organization, led by Profs. Zhou Lizhu and Tok Wang Ling and the technical program, chaired by Beng-Chin Ooi from the National University of Singapore, were excellent. It’s hard to beat the Summer Palace as a location for a conference banquet.

I wish I could give a technical summary of all the innovations reported at the conference. Instead, I’ll comment about a conclusion I drew from the three keynotes and give a quick plug for my own paper (this is my blog, after all).

SIGMOD featured 3 excellent keynotes from esteemed members of our community. Phil Bernstein from Microsoft Research talked about the progress made on Model Management, a research agenda he started 7 years ago, and on the challenges ahead. H.V. Jagadish from the University of Michigan talked about how to make databases usable, identifying some of the pain points and research challenges. Finally, Gerhard Weikum from the Max Plank Institute in Saarbruecken, Germany, talked about research combining databases and information retrieval. Each of the wrote a paper for the proceedings that is very worth reading.

I found a couple of common themes among the keynotes. First, they are all pushing the community in important and non-traditional directions. I found that extremely heartening. Second, I think the three keynotes, each from a different angle, support the claim that we need a much better understanding of users and their pain points when they work with structured data. That's a very touchy subject for database folks, who are used to spending their time 'under the hood'.

In Jagadish’s case, usability was the subject of the talk, and hence understanding users is crucial. In Phil’s case, he gave the example of generating schema mappings (mappings between disparate databases), and he was trying to get at what the pain points may be there (he argued that in the contexts he’s been considering, producing schema matches, typically the first step in mapping generation, is not longer the bottleneck). In Gerhard’s case, the question that comes up is what is the right answer when we combine DB&IR in a single system, i.e., what is the real user need. In a DB system, the semantics of the query clearly dictate the answer, but when you combine structured and unstructured data, it's no longer clear what the ranking criteria should be.

The point I’m trying to make is the following. As a community, we need to study user needs as they work with structured data, whether they are creating data, trying to understand existing data, formulating queries or creating mappings. Importantly, we need to keep in mind that a user’s task is rarely just to get an answer from a structured database. Users are typically working with both structured and unstructured data, and their tasks are broader than a single query. A useful interaction with a system is one that brings them closer to completing their task (I know, this his fuzzy, but that's why it's research).

It is tempting to push these problems to the HCI community, but I would argue this is a mistake. These problems will not be high enough on the agenda of the HCI community (there, if your device doesn’t move or perform magic, it’s uninteresting), whereas for us they are crucial for identifying good research directions and evaluating them. As a community, we need to find a way to encourage research on usability and to learn from the HCI community how to evaluate such research. We need to bring this agenda squarely into our conferences.

I'm not the only one to touch on this topic, and we're not the only community to see this need. A recent report titled “The Landscape of Parallel Computing Research: A View from Berkeley”, argues a similar point about developing novel programming models. I thin visualization is an important component of this research agenda (see Anant Jhingran’s blog post about this very point), and see Laura Haas’ ICDE excellent recent keynote and paper for a very nice articulation of this argument in the context of data integration.

My student Luna Dong presented our paper on indexing dataspaces. This paper has the distinction of being the first technical paper I published with dataspaces in its title. The paper describes a set of indexing methods that enable efficient querying of a collection of loosely coupled data sources (i.e., we do not have semantic mappings between them). Because of the nature of dataspaces, the queries we support enable the users to specify structure when she knows it, and keywords to complement the structure (we call them predicate queries and association queries). The basic idea underlying the solution is to extend the technique of inverted lists from IR to incorporate information about the structure of the data. Importantly, the technique also incorporates hierarchies in the data, and therefore it enables uniform querying data sets that have different underlying structures. Luna performed a series of experiments showing the benefits of our approach, and comparing them to techniques for indexing XML that are the closest contenders to address these problems.

Tibet -- A Feast for the Eyes

2007-06-23T22:03:00.000-07:00

Oriana and I just returned from an 8-day trip to Tibet. You can see a selection of pictures here, and the story below.

In the days before our departure, all I seemed to hear were stories about people who got very sick in Tibet because of the high altitude. In fact, in all the stories, the person who got really sick and had to be flown out was always described as “young, in his forties, and in otherwise very good shape”. I decided to ignore the stories and think positively.

Getting off the plane in Lhasa airport we felt light-headed, like waking up from with a hangover. Had to walk very slowly to the baggage claim (but still able to see the big sign with my full name held up by our guide to be). We spent the first day in the hotel, trying not to move and let the body adjust. (It’s recommended to come to Tibet clean because they advise you not to take a shower the first night as part of the acclimation program.)

By the second day, we were able to walk and even climb the Potala, and by the third day even the headache went away. It took our tube of toothpaste about the same time to get used to the high altitude.

We spent the second day in Lhasa, going to the Jokhang (the most revered religious structure in Tibet), and the Barkhor, a lively market surrounding it. Already there, we experienced first hand the devout nature of the Tibetan people. While there were some tourists, we were overrun by locals making their way into the temple to make their offerings. In other parts of the world I’ve seen little old ladies pushing their way to a bus; here they were pushing their way to the Buddhas to offer anything from barley flour and yak butter to coca cola.

We then went to the Potala, Lhasa’s best known structure, the seat of the Dalai Lamas. It indeed was as impressive as I imagined it to be (yes, from the movies). The night views of the Potala were especially impressive.

The most striking thing about Tibet (and the reason I’ve been talking about going for the last so many years) is the prevalence of color everywhere. It starts from the prayer flags erected on most houses, on bridges and peaks of mountain passes. The colors of the Tibetan clothing are wonderful. The window and door treatments, even in the poorest places are simply mind boggling. Even after a few days in Tibet I was simply amazed and happily taking it in as we drove throughout Tibet.

And drove we did (in the backseat of a Toyota Land Cruiser). Distances in Tibet are significant, and it’s not unusual to get stuck behind an army convoy or have to wait on the roadside for a high ranking official convoy to pass by. The drivers were actually reasonably careful (apparently, traffic fines are high enough). There are mileposts everywhere, so even though you’re often in the middle of nowhere, you can be quantitative about it.

The third day we drove to Shigatse, 260km west of Lhasa, home to the Tashilhunpo monastery, the seat of the Panchen Lamas for many years. (Supposedly, the relationship between the Dalai and Panchen lamas is like the sun and the moon, but there is more to the story than that). The Tashilhunpo was the most impressive monastery we saw and was full of (religious) life. On the way to Shigatse we visited Yamdrok Lake at 5000m (just when we thought we adjusted to the altitude…)

After the Tashilhunpo we realized there we cannot possibly be impressed by another monastery. We had also spoken to a few other tourists who were on their way to the Himalayas. However, changing plans in mid-trip was nearly impossible. Nevertheless, Oriana started a long and drawn out negotiation with our guide, driver and tourist agency that seemed about as complicated as a typical M&A deal she negotiates in her day job.

As the negotiations proceeded, we drove to Tsetang (east of Lhasa), the cradle of Tibetan civilization (and home to the nicest hotel & breakfast we had). After a 40km drive on a very bumpy and windy road we arrived at Tibet’s first monastery, the Samye monastery. We were proven wrong; were blown away by Samye as well.

The next morning, with the help of a promise of a tip, the negotiations came to a close and we started driving westward towards the Himalayas. We started at 7am, and drove for 12 hours through multiple mountain passes, very rural areas of Tibet and a couple of other hurdles that I was advised not to blog about. At 7pm, we were standing at 5000m elevation, looking at Mount Everest and its sibling peaks (Makalu, Lohtse, and Cho Oyu). The scene was definitely worth the drive, though perhaps one can argue that the sight of Rainier from Seattle is about as impressive. Naturally, the only other tourists with us at that vista point were a bunch of young Israelis. (Except for multiple groups of Israelis, we saw some French, some Americans and at some points, what seemed like the entire nation of Japan). At 7pm we started a 300km drive back to Shigatse, the closest place with a reasonable hotel.

Driving through Tibet we noticed that it probably has the highest per-capita number of pool tables. Pool tables were adorning the sides of the roads in the most rural villages. In some cases, they were actually used for playing pool, and in others, as stands for decorative items. Our guide didn’t offer a convincing explanation of the phenomenon. Cell phone reception, even in remote parts of Tibet, was typically better than in my home or office. That enabled me to read (and mostly ignore!) my email.

From the culinary perspective, once you get over your craving for yak meet, yak milk, yak butter, your main choices are Nepalese, Indian and Chinese food. I did manage to find a descent cappuccino in Lhasa!

I will not make any political comments on Tibet here, but will mention one anecdote. Apparently, in the past few months, two groups of American students went to Everest base camp and staged pro Tibetan independence demonstrations. As a consequence, except for making it harder for others to reach there, their innocent Tibetan guides and drivers were put into prison for 5 years. So if you’re going to make political statements, make sure you understand the local dynamics before you put your friends at risk.

In summary, Tibet is a wonderful place to visit. Although it is modernizing very rapidly, you can still see the old, especially if you head out of Lhasa. Be prepared for long drives a few dirty toilets here and there. But make sure you bring a good camera (thanks Pandu for the great recommendation!) and take it all in! I’m sure that of all people, my mother-in-law Helen is the happiest we made the trip because now she doesn’t have to hear me talking about going there anymore.

SigTube: 5 Minute Presentations of Conference Papers

2007-06-02T21:55:00.000-07:00

The success of YouTube and the like have proven a very simple point: a 5-minute video is a very effective mode of communication. People love to see stuff in 5-minute nuggets (or less). I'm suggesting we learn from this observation for better dissemination of scientific results.

I'm proposing that along with every paper published in conference proceedings, we also create a 5-minute video presenting the highlights of the paper, and make the presentations available on the web for free. I'm calling this SigTube (mostly to encourage people to come up with a better name).

A 5-minute presentation (done well) can give quite a bit of information and insight about a publication, certainly more than the 100-word abstract or the paper's introduction. I know I would love to sit through a bunch of these from time to time and learn more about what's going on in my field, even in areas that are farther away from my main interest areas (in fact, probably mostly in such areas). A video also captures the enthusiasm and emphases of the speaker (not to speak of the fact that it preserves their youth for eternity!)

With today's infrastructure and technology, this is pretty much trivial to do. A conference can dedicate a person with a video camera who will film the videos during the conference. Alternatively, some authors may prefer to film the videos on their own and send them in.

Any takers?

Me and Web 2.0

2007-05-27T21:57:00.001-07:00

My Web 2.0 credentials are really shooting through the roof as of late.

As of yesterday, I uploaded my first video to YouTube. The video shows my daughter (6 y/o) dancing nicely, with my son (18 months) "accompanying" her. As you can see, my son has already attained my level of dancing ability (my daughter passed me a long time ago).

I decided to try out Google MyMaps for fun. I created a map of my life and travels (and had fun doing so). Take a look -- (with the subtle implied hint that I'm happy to be invited to places not yet marked on the map).

Finally, together with Sihem Amer-Yahia from Yahoo!, I'm organizing a panel at VLDB 2007 (Vienna, September) on "Web 2.0 and data management". We have an exciting lineup of panelists that includes Anant Jhingran from IBM (who also blogs furiously), Gerhard Weikum (Max Planck Institute in Germany), Donald Kossmann (ETH Zurich) and AnHai Doan (U. of Wisconsin, Madison). I'm sure I'll be saying more about this panel on this blog, so stay tuned.

Circle of Blue - Eloquent Version

2007-05-27T21:48:00.000-07:00

Keith Schneider wrote a much more eloquent post about the Circle of Blue Powwow I went to a couple of weeks ago.

This is a (perhaps very rare) opportunity to directly compare the writing skills of a guy who regularly writes for the New York Times with those of a guy whose readership includes mostly database professionals.

Circle of Blue

2007-05-12T21:36:00.000-07:00

I spent Friday with an amazing collection of people mostly from the journalism and photography world. Among others, this collection included a previous photography director for Newsweek and Sports Illustrated, a creator of events such as the opening and closing ceremonies of the Salt Lake City Winter Olympics and the 50th anniversary of Disneyland, authors and writers for various newspapers, picture editor for the Washington Post, the person responsible for managing the US government water policy, a previous director of multi-media for MSNBC, founder and director of the Pacific Institute, and a NASA astronaut. Some of the people attending had spent significant amounts of time in third-world countries working on sanitation projects.

We were all hosted on the Pine Hollow estate, which is an amazing 30+ room mansion on the shores of Lake Michigan, a bit north of Traverse City. The home was built by Leslie Lee, and includes every amenity imaginable to man combined with excellent taste in design.

So what we were we all doing there? This was basically a Circle of Blue powwow. Circle of Blue is a non-profit organization that is dedicated to raising the awareness of the public and policy makers to the diminishing supplies of clean and affordable fresh water. CoB tries to raise awareness through a combination of journalism, photography, film and data collection. Carl Ganter, the founder, is quite an amazing guy and among his other major accomplishments (e.g,. being a photographer for National Geographic) tells a great story of how, through a great work of photography and journalism, he (and others) were able to exonerate a wrongfully convicted father and reveal the real murderer in a case in Illinois. He and his wife Eileen conceived and planned Circle of Blue.

There is no way I can do justice to the entire discussion in a short blog post, nor can I fully convey the tenacity and passion of the people gathered. I will also skip the many details on the major water issues facing our planet (but I will point out that water is one of the few main foci of Google.org, the Google Foundation). Instead, I'll just highlight a few points I found interesting from my perspective.

Our discussion focused on how exactly to leverage tools and technology to raise awareness on water issues. The ideas discussed were all over the map. They ranged from creating blue rings that everyone would put on their faucets (following Lance Armstrong's yellow rings for cancer fighting), to using Web 2.0 tools such as blogs, Google My Maps, Flickr, etc. to help people all around the world to create databases of water-related issues, and to mobilizing the religious right to take up their issue in their congregrations.

In a sense, we were trying to figure out how to recreate the success of the green movement, but in blue. While there is much in common between the two global warming issues and water issues, there are also a few key differences between the two. First, in the case of green, there are some simple things everyone can do to help a global problem (e.g., buy a hybrid, go solar). In the case of water, aside from taking shorter showers and watering your garden more effectively, many of the major issues are of local nature and the problems and solutions vary quite a bit. Second, the people suffering from water shortages at this point are typically far away and that makes it hard for the issue to be on people's minds constantly. New Orleans is much closer to home.

The other interesting point about the discussions was how to combine traditional media like journalism, film and photography with newer technology to create viral awareness of the water issues. While it's great to have the high-quality polished artifacts created by these media, we also need the bottom-up YouTube-type videos and blogs created by a much broader and geographically distributed set of people, but with much less skill (myself included...) to really reach people's attention. We need to collect good data, but mostly, make sure the data is used in effective ways for highlighting the issues and garnering world-wide attention.

This was a highly inspiring meeting for me. If you have any ideas, don't hesitate to post a comment, send me email, or contact Carl Ganter directly. I'm sure this topic will reappear on this blog.

The Slash Effect

2007-05-12T20:53:00.000-07:00

I just finished reading "One Person/Multiple Careers", by Marci Alboher, Author/Speaker/Coach. Like with other books described in this blog, my reading agenda is highly influenced by authors visiting Google (which happens often enough to keep any reader quite busy).

The main contribution of this book is to get you thinking. Slashers are people who have multiple parallel careers. Through numerous examples, the book claims that this is a growing phenomenon in today's culture, and describes the challenges, opportunities and benefits having to do with slash careers. The point that I found most interesting about all of the above is that slashing essentially gives you multiple identities in society. Think of what you answer at a party when people ask you what you are or do. Being a slasher means you can give multiple answers, or choose one you think best suits the situation. But more than that, slashing means you gain some internal balance in life, rather than being tied to one professional identity.

Marci gives examples of lawyers turned writers & coaches (including herself), a teacher with a modeling career, a computer programmer who also directs a theater, a lawyer who's also a Baptist minister, Sanjay Gupta, the CNN health correspondent who also does surgery a few times a month, and the list goes on and on. She discusses how people manage multiple careers, some of the cross-over benefits and life-style benefits they obtain, and she offers practical advice on how to become a slasher. The book essentially revolves around all these examples, and every chapter ends with the highlights of its main points (great for future reference).

Being a somewhat formal guy on occasion (perhaps one of my slashes?) I found myself looking for a definition of a slash. Marci seems to focus on aspects of life that are part of your career (it doesn't actually matter whether you derive much income from it, otherwise most of the poets and actors would not have made it into the book). But, for example, does a hobby count as a slash? Does it depend on how much time one spends on the hobby? In fact, many jobs are composed of multiple slashes (e.g., professors spend half of their time teaching, half their time doing research, and the other halves raising research funding and sitting on committees).

Clearly, parenting is the most common form of parallel activity adults engage in. The book contains a chapter on parenting and how parenting and slashing share many challenges. The book even claims that a slash life can prepare you better for parenting (though clearly, some of the slashes may take a back seat for a while).

However, my search for a formal definition of slashing is missing the point. As I stated at the outset, the point of this book is to make you think about all the aspects of your life whether they count as slashes or not. Personally, the most common slash combination I've encountered (and personally experienced) is the professor/entrepreneur combo, and I can speak at length about the benefits and challenges there.

Finally, one point that was not addressed in the book is multiple careers that happen in sequence, rather than in parallel. Perhaps I'll take the opportunity to coin a new term: the double backslash, (for those of you who haven't had the pleasure of using the Latex word processor, I should explain that a double backslash creates a new line in the text). I would think that slashing and double backslashing share many of the challenges and benefits.

In summary, this book is a rather quick read (you can skip parts, but pay attention to the boldfaced sentences). I found myself reflecting on my slash/double blackslash riddled career and wondering what other slashes may come into my life at some point.

Chinese poetry recital

2007-04-29T22:07:00.000-07:00

This post is a few weeks late, but the parental pride is not diminished at all. My daughter Karina participated in the Northern California championship for Chinese poetry recital. This competition is organized by the association of Chinese schools of Northern California that includes over 70 schools and includes many categories (remember: this is all Chinese, so I'm a bit sketchy on some details). To get to the regionals, she had to win her school competition. She was given several poems and in the competition she had to recite one of them. The competitors were judged mostly on pronunciation (and on not using their hands).

There were 21 participants in her category (she competed in the 5-7 age group, being on the very younger side of that). Parents were not let in the room or even to see the judges before the competition.

As the picture shows, Karina took First place! The victory was immediately celebrated by the biggest ice-cream she ever had, but suffice it to say that her maternal line (i.e., mom and grandmother) did not sleep that night of sheer excitement! Clearly, this is one of Karina's achievements that I made absolutely no contribution to.

Pandu's blog

2007-04-29T22:00:00.000-07:00

My good friend and grad school buddy Pandu Nayak recently started a blog. He has a higher ppd (posts per/day) than I do and the posts are on a variety of topics (you'll never get bored).

Pandu and I often IM each other (mostly for coordinating critical issues such as espresso consumption). I think the next step is for us to get MySpace accounts, and then we'll be Web 2.0 compliant. Who knows, maybe with such openness, our children will consider talking to us when they become teenagers!

From Darfur to Robotics

2007-04-29T21:32:00.000-07:00

FIRST is an international competition that engages teenagers in technology. Their best known competition is in robotics, but they have a bunch more. My connection to FIRST is through coffee (I had the pleasure of meeting the founder of FIRST, Dean Kamen, next to the espresso machine at Google), and through family (my brother-in-law Asaf Menuhin has been running the FIRST competitions in Israel). The following inspiring story appeared in an Israeli newspaper and was forwarded to me by my parents.

"A" is a 16 year old student who escaped the killing in Darfur two years ago. He somehow got to Egypt, and from there crossed the Israeli border. Initially, he (and the others with him) were arrested. After a while, he was let out and joined the Kfar Yamin boarding school not too far from Haifa, Israel. "A" immediately became a star student at the school. One night, when he was walking around the school he noticed the lights on in the science lab. He went in and saw a group of kids preparing for the FIRST regional robotics competition, and immediately fell in love with robotics. Shortly after, he became the leader of the group. "A" led the group to an impressive 4th place standing in the Israeli regional. He was disappointed because that was not enough to earn him a trip to Atlanta for the finals, but everyone else is still in awe of the huge step this young man made in such a short time. I bet Dean did not anticipate this story when he started this amazing establishment!

Memorial Day

2007-04-22T21:29:00.000-07:00

This is a somewhat unusual topic for this blog, or perhaps a sign that I'm finally getting with the spirit of Web 2.0.

Today is Israeli Memorial Day. We remember the soldiers and civilians that were killed in wars and terrorist attacks during the history of Israel. In Israel, this day is taken very seriously (e.g., the notion of a Memorial Day Sale does not exist). As a striking example, at 11AM on this day, there is a siren sounded throughout the country. Every person, and I mean, every single person, will stop what she or he are doing and will stand with respect for 2 minutes. If you're driving a car, you stop the car and stand outside. 2 minutes of complete stoppage. Immediately after the siren, the memorial services begin at all the cemeteries. In the evening, as the sun sets, the country turns from a day of mourning to a day of celebration of its independence. It's quite a striking transition.

For me this is always a very special day. My middle name, Yitzchak, is in memory of my uncle (my father's brother) who was killed in 1948 during the war of independence. He was 27 when he was killed, serving in the Israeli Air Force. I, of course, never got to meet him, but indirectly, he influenced my life quite a bit. He inspired my father to get into chemistry, which took him to academia. In growing up, I never questioned whether I'd end up as an academic or not.

Since moving to the Bay Area, I've been attending the Memorial Day ceremonies organized by the local Tzofim (the scouts). As part of the ceremony, they read out the names of fallen ones who have relatives in this area. They read the names in chronological order of their deaths. It struck me as I was sitting there tonight that I was sitting in anticipation to hear the end of the list, i.e., the new additions from last year. Unfortunately, due to the war in Lebanon in the summer of 2006, the list was indeed longer, and their stories heartbreaking as usual. Let us all hope these lists stop growing longer. There is too much unnecessary pain already in the region.

Wikinomics

2007-04-08T22:23:00.000-07:00

I just finished reading Wikinomics, by Don Tapscott and Anthony Williams. This is certainly an interesting and thought provoking read. It's interesting from various perpectives, including the structure of current and future businesses, the future of education, and how to think of one's own career (at whatever stage you may be). Certainly, as with many books of this sort, the same message could have been conveyed in fewer pages (this applies especially to the first 40 pages or so), but overall, it's very worthwhile.

The main thesis of the book is that mass collaboration changes some fundamental aspects of running a business. Three forces have recently come together to create the perfect storm that facilitates Wikinomics: (1) technology (basically, Web 2.0 where anyone can contribute to anything), (2) the Net-Gen -- the generation of people who grew up collaborating (think: kids who view email as a thing only their parents do), and (3) the global economy, where companies are forced to reach out and collaborate to produce additional value (i.e., The World is Flat).

Perhaps the most succint description of the principle underlying Wikinomics is a rephrasing of Coase's Law (which was coined around 1937 by an English socialist). The law says that a firm will expand until the costs of organizing an extra transaction within the firm become equal to the costs of carrying it out on the open market. For example, if you're a car company, if it's cheaper to manufacture your own tires than use an external supplier, then you will do so in house.

The observation is that the internet has lowered the transaction costs so significantly, that now the right way to think of Coase's law is that nowadays firms should shrink until the cost of performing a transaction internaly no longer exceeds the cost of performing it externally.

The book then goes on to illustrate examples of the different aspects of Wikinomics:

Peer production: examples of great achievements created by large collections of collaborating peers: Wikipedia, Linux (and more importantly, IBM's embrace of the open-source community as an example of a firm doing the right thing).

Ideagoras: essentially, using the open market for research into your specific problems. The observation being that no matter how many researchers you employ, the person with the best ideas for a particular problem is likely not in your lab (the authors still emphasize the importance of internal R&D though). Here, the main examples are InnoCentive, a company that acts as an eBay for ideas, and Proctor & Gamble, that was in quite a bit of trouble, but managed to tap external ideas to make a comeback.

Prosumers: companies that benefit from their consumers essentially developing their products. The big example here is SecondLife, where the consumers create more than 99% of the content being transacted (and the consumers get the IP rights to anything they create!) Another example is Lego that lets users create and share their Mindstorm creations. An interesting example is that of consumers tinkering with iPods (and Apple not standing in the way, as opposed to Sony not taking that approach).

The New Alexandrians: the creation of new data banks that enable research to proceed faster (the Human Genome Project). One of the interesting discussions there was about reshaping the relationships between universities and companies. The recently created Intel Lablets are an excellent example of that (and I predict this is one we'll see more of).

New platforms: Google Maps, need I say more? Ok, I will. Actually, Amazon was way ahead on creating platforms for others to build on, and today Yahoo, eBay, Google and Amazon are creating exciting platforms for others to create additional services on.

The Global Plant Floor -- companies changing the way they interact with suppliers. Instead of Boeing sending exact specs for each part of a new plane, they let the suppliers design and innovate as much as possible. They also let them assemble much more of the components, and therefore, it now takes Boeing 3 days to assemble an airplane once all the parts have arrived, rather than 30. BMW is another example described here, along with Lifan, a Chinese company that is making waves in the motorcycle industry.

Finally, there is a discussion of how wikis really changed the possible interactions in corporations.

A few thoughts.

First, there is no doubt that there is a lot of evidence of the power and presence of Wikinomics presented in the book. While the examples were very good, they were still few. This leaves one with the feeling that Wikinomics may remain a fringe rather than mainstream in business.

Second, one may wonder whether we didn't hear all this in The World is Flat by Tom Friedman. Certainly, there are many interesting relationships between the two books. I think Friedman addressed a narrower aspect of the picture: basically that companies can distribute themselves across the world and work more aggressively with partner companies. Wikinomics goes one step further and discusses how companies should leverage the masses, not just partners, and how that affects the way we think of IP rights and communications within corporations.

Third, as a computer scientist, I wonder what CS has done about all of this. While we've been responsible for many of the technologies that created the tools mentioned in the book, I'm not sure we leveraged the tools ourselves to benefit our own research. Certainly, in education we haven't. In fact, Don Tapscott argues that are very few industries that have changed very little in the past century, but education is one of them. It's still mostly built around teachers standing in front of students and lecturing. Food for thought.

Finally, there is a lot of discussion in the book on IP issues and generally, on how business relationships between companies should be structured. I think every lawyer should read this book, and hence, am moving the book to my wife's side of the bed.

The J Curve

2007-03-10T13:37:00.000-08:00

I've decided to start writing short summaries of (some) books I read. Please do not expect literary pieces here -- this is not my attempt to sneak into the book review section of the New York Times, nor is it an attempt to make up for a few missed book reports in grade school.

Had I started this two years ago, I would have definitely blogged about "The World is Flat" by Tom Friedman. Too late now. But I think this book should be required reading for anyone interested in data management (either in research or industry). Just read that book and imagine all the data management services we need to invent to support the flattening of the world. Of course, the book has other merits too.

The J Curve by Ian Bremmer, is essentially a framework for discussing the stability and openness of countries and how you can understand political events in the context of moving on the curve. You need to imagine the letter J rotated about 45 degrees clockwise to get the full effect. The X axis of the J curve is the degree of openness of a country (e.g., travel restrictions, freedom of the media, economic openness, the presence of independent political institutions). The Y axis is the degree of stability of the country (i.e., whether certain events would cause great chaos or not). If you're on the top left of the curve, you're closed but stable (e.g., North Korea). If you're on the top right, you're open and stable (e.g., USA, western Europe). If you're China, you pose an interesting challenge to the curve (more on that in a moment).

One of the main points the book makes is that for countries to go from the left to the right they will have to first go down the curve and therefore suffer some considerable instability. The world can help these countries by raising the entire curve, i.e, make the depths of the curve stable enough so countries will survive the transition. (In practice, that observation lets Bremmer criticize many of the policies the USA has taken w.r.t. some countries).

The book goes through a few examples of countries in each part of the curve. It starts with North Korea, Cuba and Saddam's Iraq as examples of stable but closed. It discusses Iran, Saudi Arabia and Russia as countries that have the potential of sliding down the curve from the left. He shows South Africa as an example of a country that made it through the transition successfully and Yugoslavia as one that didn't. He takes Turkey, Israel and India as examples of countries founded on the right hand side of the curve and who have maintained it that way (though they do face challenges going forward). Finally, there is a chapter on China, where Bremmer argues that despite its economic openness, China is still on the left side of the curve.

What I liked most about this book are the brief yet insightful summaries of the relevant history of each of the countries discussed. The summaries give you the background for why things are the way they are now and let you understand better the challenges facing the countries. I'm finding that it's easier for me now to put current news into context, and in fact, that the curve does give a pretty good framework for thinking about today's world events.

That being said, the chapter on the country I am most familiar with, Israel, was a bit disappointing, so perhaps people from other countries would say the same about their respective chapters. I enjoyed reading about all the complexities in Yugoslavia, though I wished he would have spent a page or two describing some of the main events of the war there (he stopped just before it saying it would be too complicated).

In discussing the book with Donald Kossmann, he wonders whether corporations can also be classified according to their openness. For example, a company who has a very closed set of programmatic interfaces to their products (and I won't name the names he mentioned) may be considered a North Korea of countries. An interesting thought to develop.

In summary, I would definitely recommend this book. If you're an expert on world affairs and attended all your history classes in school, you may find less of a payoff from reading it.

I'm moving on now to Wikinomics.

Uncertainty and data integration

2007-01-29T22:03:00.000-08:00

A short rant about the relationship between databases that manage uncertainty and lineage and data integration systems.

Recently, there has been renewed interest in building database systems that handle uncertain data and its lineage in a principled way. The Trio Project at Stanford and the MystiQ Project at the University of Washington are just two examples (I collaborated for a while on the former, and watched the latter up close while I was a professor at UW).

I think this is a great research area and certainly (no pun intended) a very timely one. I want to make two points though (one of which may raise Jennifer Widom's blood pressure).

First, I think data integration is the killer application of uncertainty and lineage (ok, maybe there is a second -- sensor networks). Fundamentally, data is uncertain when it comes from external sources and some of the transformations it went through on the way are not necessarily correct.

In fact, I think one of the greatest challenges for data integration research is to build data integration systems that deal gracefully with uncertainty (uncertainty can be about the underlying data, the schema mappings and the mapping of keyword queries to structured queries). If you have good ideas about this, please do contact me.

My second point is that there is really no argument here. In fact, I believe that once a database system is able to model and process uncertain data and its lineage, much of the distinction between traditional database systems and data integration systems goes away.

Specifically, by modeling data lineage and that it may be uncertain, you're admitting that the data came from somewhere, and that you're not sure about the transformations the brought it into the database or about its intrinsic meaning. That's exactly what data integration is about -- modeling data that comes from multiple sources. Unlike ordinary databases, where the data might as well have been born in the database because you know nothing about its past, databases with uncertainty and lineage admit that data had a prior life.

So then what's left of the difference between databases and data integration systems? Mostly issues having to do with query processing over remote sources.

I should emphasize -- I'm not claiming that these problems are solved (quite the contrary, see my comment about about data integration with uncertainty). But I do find it quite appealing that a database system models the fact that data came from the outside. That's the way it typically is in the real world, and it's about time databases realize it too.

Structured data and the web

2007-01-29T21:49:00.000-08:00

One of my main areas of focus at Google is on the relationship between structured data and web search. There are now vast amounts of structured data out there, mostly in the deep web (i.e., in databases behind HTML forms), created by annotation schemes (e.g., Flickr, Google Co-op, etc), and Google Base. The question then is how to use this data to improve the results of web search.

I recently published two related papers on this topic, one at CIDR 2007 and one in the Data Engineering Bulletin (go to Page 19 of the issue). You can read the papers for the details, but I'd like to highlight two key points from these papers that should be kept in mind when researching this area:

Integration: Whenever you cook up an idea about how to improve web searh by leveraging structured data, or by automatically structuring data on the web, you need to keep in mind how your technique will integrate with other web searches. Users want to go to a single search box to find all their result. So whatever technique you come up with, needs to mesh well with other techniques used by the underlying engine.
Data about everything: Many ideas work well if the domain of the data is constrained (e.g., you know you're building a portal to search for cars, housing or job listings). But on the web, data is about everything. There is no domain or set of domains that covers all data on the web. In fact, it's not even clear when one domain ends and another one begins. So try to imagine what it's like to deal with data about everything. That changes a lot in the way you think about a problem!

Ein Gedi

2007-01-29T21:40:00.000-08:00

I recently went for a quick trip to visit my family in Israel (with a short stop in Amsterdam, where I saw an amazing exhibit).

While in Israel, I went for a hike in Ein-Gedi, one of my all-time favorite places. The trip was organized by the Tova Milo's database group at Tel-Aviv University. Check out the pictures from Ein Gedi (and a few others).

While Ein Gedi has always been a place for me to find complete peace, staring into the Dead Sea, this time my blackberry made it a slightly different experience.

Introduction

2007-01-28T11:50:00.000-08:00

I've finally decided to start blogging and am very excited about it!

The posts on this blog will either be about work (i.e., data management ideas) or my family. No politics (you probably don't want to hear it anyway), but possibly a passing comment on coffee or other exciting events.

In way of background, until recently I've been a professor at the University of Washington. I moved to Google in September 2005 and lead a group looks at how structured data can be used in Web search. As I publish papers about this work, I'll summarize them here. For all my publications prior to coming to Google you can check out my UW web site.

One of the goals of this blog is to get people in the data management community to share novel ideas and discuss them. While technical results are well served by pubished papers, Web 2.0 gives us the opportunity to discuss ideas outside our conferences quite easily.