Wednesday, July 14, 2010

How to think about data?

Recently, I have been working on characterizing an optical system that I designed two years ago.  It works and served the purpose for which it was designed.  During the design and experimental data acquisition I observed a curious effect that I didn't intended it to have.  It took me a while to gain intuition about what is happening.  Now with the help of my colleague, Indranil, we have created a setup where I can capture all the data I need.  But I still don't have clear understanding of how to think about the data and how to present it.  I wish I were as comfortable visualizing the 4D and 5D data as I am with 2D and 3D.  I intend to pursue this further and figure it out.

I realized the bigger question while thinking about this problem. With the rapid advancements of technology, we are able to design novel systems and collect tons of data but our understanding at large hasn't evolved as rapidly to make sense of these data.  Robert Sapolsky put it very succinctly in his refection to Edge's "what will change everything" question. "...The problem, of course, is that we haven't a clue what to do with that much data. By that, I don't mean "merely" how to store, or quantitatively analyze, or present it visually.  I mean how to really think about it..."

Better data visualization helps in gaining greater insights into the problem that often times results in the generation of new ideas. 

No comments:

Post a Comment