Posts Tagged ‘Databits’

Dataviz needs good UI tools

Tuesday, May 12th, 2015

A few weeks ago I set out to participate in a geospatial visualization challenge. It was exciting to finally get a chance to make an interactive map with more than markers and basic mouseover effects. It was also intriguing to try and display uncertainty on a map.

But, even the basic visualization below took more time than planned.

See databit Geospatial Prototype by Sergiy Nesterko on Databits.

What I realized very soon after starting my work was that I was spending 80% of the time implementing the user interface for my visualization. This UI experiment, for example, combines a tooltip with clickable menu. It took the most time and still isn't well rounded!

This is a real problem. I want to think about how to make an aesthetically pleasing and effective visualization, and not endlessly debug a misbehaving UI element.

Can the lack of good open source UI elements be a limitation for dataviz? Are makers of visualizations constrained in their creativity because of the lack of good UI elements? I set out to see about that. Here is a competing challenge entry by Quinn Lee:

See databit medical geospatial by Quinn Lee on Databits.

Here Quinn uses a very nice generalized cross filter library called DC.js.

The main UI elements of Quinn's data visualization are the plots themselves -- you can click on their parts, zoom in/out on maps, and highlight a range of the time series plot. All these interactions will change the way the plots look. This is great. The only caveat is that this UI pattern is native to DC.js and is not very generalizable should you decide to say replace the maps in Quinn's visualization with maps built with d3.geo.

Here is another challenge entry, by Moritz Winger.

See databit GeoSpatial Challenge by Moritz Winger on Databits.

I really like the colors and style of the piece.

In terms of UI, you can interact with the visualization by hovering over different elements, and clicking some of them. The question is this: would a rich open source UI element library help Moritz make an even better visualization?

I would like to see these elements available when making an interactive visualization:

  1. Good tooltip menus for hover effects. D3-Tip is a great start and no wonder it gets o much usage and development. But, it does not always work
  2. Good contextual menus. That is what I am working to implement with my TooltipMenuHybrid class in the first visualization in this post
  3. Good open source libraries for positioning your plots in a mobile-friendly layout, and to allow users to drag and resize them. Again, my LittlePlot class above is one such experiment
  4. Libraries for keeping the state of the visualization in the URL bar
  5. ...and so on

That said, it has certainly become easier to analyze data and convey data-driven insight today than ever before. The growth in open source
tools and related resources for dataviz isn't slowing down. That means that the set of UI options available to creative coders will keep expanding and the bottleneck I describe here will soon go away.

Responsive design for data visualization

Monday, March 2nd, 2015

Interactive data visualization is an increasingly important tool of conveying analytic information. It is used in teaching, in product design, data journalism, scientific research, and many other fields.

Much of the analytic insight that we consume every day goes through the minds and hands of data scientists. Data scientists own data analysis, this is our profession. Sometimes, we need to create interactive visualizations to communicate our work.

Our audience is increasingly diverse: analytics product owners, researchers, journalists, general public are just a few examples. How do we make sure our visualizations are maximally understandable and usable by our target audience?

In order to make interactive visualization more usable, data scientists need to start thinking a little bit like designers. Yes, it means expanding the significant skill set we already possess even further, or partnering with a designer. With the abundance of information and tools currently available, it's not that hard -- and even fun.

A cornerstone of good design is responsiveness. The interactive visualization we make needs to look good on any screen width. Here is an example, an interactive dashboard of edX learner progress by Zach Guo:

Zach's visualization adapts to different widths of the containing box. See it for yourself by clicking and dragging the resize handle on the bottom right of the visualization:

Here is another example, a network visualization I made some time ago. It also adapts to different screen widths.

If your visualization is web-based (like both visualizations above), most of the time you can make it responsive by defining different CSS layout rules based on screen width. Here's how I did it in the visualization above (check out source code for a complete picture of what's going on):

@media (min-width: 1150px) { {
        width : 250px;

There is more discussion on data visualization usability in the material of a course on statistical computing and interactive visualization I taught at Harvard Statistics department in Spring 2013 (for example, consider Lecture 24).

Data scientists own data analysis. Part of making the end result accessible to the target audience, responsive design for data vis can be a fun addition to our toolkit.

Effective way for data scientists to grow impact

Sunday, October 5th, 2014

In order to get things done people need to communicate effectively. At school, teachers present to students. In consulting, consultants make powerpoint slide decks. In research, researchers make presentations and talks to spread their ideas.

When it comes to data scientists, many of us write code (in R, Python, Julia etc) in order to analyze data, inform decisions. To many people, what we do is rocket science.

What is the most effective and easy way to spread our ideas and grow impact? A good answer is interactive visualization. And not just for data scientists, but for anyone working with analytics.

Sure enough, pretty and intuitive graphics are a good way to deliver insight. And, with modern technologies interactive visualization can grow into products, viral marketing campaigns, and journalism pieces.

I have been doing interactive visualization for a while. Below is a visualization I made to explore geographic enrolment patterns of HarvardX. What started as an exploratory project ended up as a product -- an interactive analytics platform we called HarvardX Insights. It ended up on the cover of Campus Technology, and several universities from around the world contacted HarvardX to get the code.

See databit HarvardX Certificates World Map by Sergiy Nesterko on Databits.

And here is something for data scientists -- a visualization of the Hamiltonian Monte Carlo algorithm. I taught it to my students last year during a graduate course on statistical computing and interactive visualization at Harvard Statistics. This visualization was one of several I created for the course together with students.

See databit Hamiltonian (Hybrid) Monte Carlo by Sergiy Nesterko on Databits.

People who work with data increasingly need to acquire and apply creative coding skills in order to put their ideas to work. This helps come closer to the end user of an analytic insight, and avoid possible operational distortions and dead ends along the way. That's why, resources that promote and teach creative coding are in high demand among my peer data scientists. I am a big fan of Mike Bostock's Blocks, and other resources such as Codepen, JSFiddle, and Stack Overflow.

Recently, I have been using and contributing to Databits more and more. Databits is a website for data scientists, data journalists, and other creative coders to share work, connect, and grow impact. I believe that eventually, the site will allow to be more targeted and specifically learn from and follow peer data scientists and other creative coders who are focused on producing effective interactive visualization and other cool stuff. For example, I look forward to learning some Processing applications from this guy. In the meanwhile, I helped put together a simple databit based on Processing.js:

See databit Processing.js Hello World Sketch by Sergiy Nesterko on Databits.

The site also runs Challenges, an initiative aimed at finding meaningful problems for creative data scientists to solve, and put on their portfolios. I find this pretty cool.

I look forward to learning new things, finding cool problems, and making the world a better place with data. Now my creative work has a home -- you can check out my creative endeavors and interests on my Databits profile page.