How to think about data?

Recently, I have been working on characterizing an optical system that I designed two years ago.  It works and served the purpose for which it was designed.  During the design and experimental data acquisition I observed a curious effect that I didn't intended it to have.  It took me a while to gain intuition about what is happening.  Now with the help of my colleague, Indranil, we have created a setup where I can capture all the data I need.  But I still don't have clear understanding of how to think about the data and how to present it.  I wish I were as comfortable visualizing the 4D and 5D data as I am with 2D and 3D.  I intend to pursue this further and figure it out.

I realized the bigger question while thinking about this problem. With the rapid advancements of technology, we are able to design novel systems and collect tons of data but our understanding at large hasn't evolved as rapidly to make sense of these data.  Robert Sapolsky put it very succinctly in his refection to Edge's "what will change everything" question. "...The problem, of course, is that we haven't a clue what to do with that much data. By that, I don't mean "merely" how to store, or quantitatively analyze, or present it visually.  I mean how to really think about it..."

Better data visualization helps in gaining greater insights into the problem that often times results in the generation of new ideas. 

To inform or to intimidate ?

"...When we understand that slide, we'll have won the war,' General Stanley McChrystal, the US and NATO force commander, ..." 




This reminds me of some of the papers and presentations in research community.  A power point slide would have so much details in there that it ends up overwhelming the audience instead of educating.  This is a result of poor authorship.  Either you don't understand your own material good enough to decide what is important and what is not (which reflects poorly on you) or you are not confident about your work and just overwhelming/intimidating people with such slides into believing that your work is good (which is even worse).  Oh there is another, equally sad, trick researchers in Engineering employ, i.e. to fill their presentation/paper with inane amount of math without providing enough intuition  and insight  behind it.  It serves only one purpose: impede the communication of the idea.




Goal of the presentation should be to inform and educate and not intimidate and overwhelm.   If you do your job well audience would be able to appreciate it.  But sadly that's not the case always.  In fact, I have seen the scenarios where people berated the idea because the presenter explained it very well.  Their argument being I understood it in 20 minutes so it may not me that difficult.  Seriously?  Instead of appreciating the author for job well done you gonna blame him for making things comprehensible.  They would be all impressed if you show some complicated (and unnecessary) math equations that might not make any sense at all unless you spend several hours at it.   As the saying goes... "if you can't convince them, confuse them." Using Math as the crutches is an old trick that has become a habit now.  I promised myself never to use math unless it is necessary in communicating the idea.