search by tags

for the user

adventures into the land of the command line

d3 for data visualisation

for about 2 years, on and off, i’ve tried to get my head around d3.js. it’s a really funky javascript data visualisation library written by mike bostock. it creates styled and interactive svg elements from datasets you provide. i really like what you can do with it, but at the same time, the learning curve, for me at least, has been significant. you can do some very complicated designs with d3, it even has its own physics engine. but im a n00bie, so i’ve only worked with the basics. to include it in your html, add this

<script src="//d3js.org/d3.v3.min.js" charset="utf-8"></script>

general learnings

different style charts require different code. you can’t just copy paste stuff, change a few things and expect it to work

coordinates

d3 draws everything starting from the top left hand corner of the page. so 0,0 means 0 pixels from the top and 0 pixels from the left. by contrast, the cartesian coordinate system starts from the bottom left. d3 will not automatically translate this for you, you need to do it yourself by inverting the height

layering

d3 utilises svg for most things. svg has no layers or other concept of depth. the order in which elements are drawn on the canvas determines their stacking order. the last painted object will appear in front of all others

defining vs adding

the code can look quite long winded, but the two main things that are happening is defining or declaring variables, and then actually adding them to the html so that they are visible to a human being. if you define but not add, nothing will appear, and if you add without defining, d3 wont know what you want it to do.

selecting and appending

to get d3 to draw on your funky website, you need to provide it with a reference of where to draw. the way that works for me is to create a html div with a class like ’.graph’ or whatever you want. then in d3, you use the

d3.select("body").select(".graph").append("svg:svg")

to select it and add the stuff you want to it. sometimes if you haven’t done this select and append properly, even though the rest of your code is accurate, you wont see any chart!! the chart is just being drawn somewhere else off the screen or behind some other element. check this is being done properly if you encounter no errors in your javascript, but can’t see any charts

data structure

decide on the form of your data structure. this is crucially important, as it will determine how d3 accesses your data. in a lot of the examples on the d3 website, data is included as a .csv file or something similar that i have never used as a method to access data on a webapp. for me data will always come in a variable containing the data as read from a database. getting past this hurdle of understanding how to construct your data so that d3 accesses the data you want is the biggest step in getting it to work as expected. so many of my facepalm moments were caused by d3 not reading the data properly because of how i structured it, or how d3 thinks i’ve structured it

line chart example

in this example, the dataset is in the form of a python dictionary (hashmap in other tongue). it is also crucially important, that if you are working with dates, that you should provide them in the form of an epoch timestamp, rather than a date string or datetime type, because javascript makes it much much harder to work with these types than it is in python

var data = {'date':'1449798361','amount':'100',
            'date':'1449798374','amount':'-200',
            'date':'1449798522','amount':'38.75',
           }

‘define’ the area of your canvas, in pixels, that you would like to use for the chart. you can put explicit numbers or use javascript or jquery to get the values for you dynamically, if say you’ve defined the div width/height as a %

var Width = 500;
var Height = 500;

or

var w = $(".graph").width();
var h = $(".graph").height();

'define’ scales. scales normalise your data to 'scale’ to the size of your canvas. so for example if your dataset goes from 0 to 1,000,000, scaling it to the canvas we created above would make the 1,000,000 data point appear at pixel 500 in your canvas. also, there are different types of scales you can choose from, linear, time, logarithmic, etc, etc.

var xScale = d3.time.scale()
                .domain([d3.min(data, function(d) { return d.date; }), d3.max(data, function(d) { return d.date; })])
                .range([0, w]);

var amount_array = []
            for (var key in data) {
                amount_array.push(data[key].amount);
            }
            var amount_max = Math.max(...amount_array)
            var amount_min = Math.min(...amount_array)

var yScale = d3.scale.linear()
                .domain([amount_min, amount_max])
                .range([h, 0])
                .nice();

'define’ axes. most charts have some sort of axis that illustrate for the reader what information is being displayed. the axes are scaled representations of an svg line that has a length defined by the scale you previously defined. in fact everything you draw from now will use your scales to map the dataset to your on screen canvas

var xAxis = d3.svg.axis()
                .scale(xScale)
                .orient("bottom")
                .ticks(20)
                .tickFormat(d3.time.format('%d. %b'))
                .tickSize(6, 0);

var yAxisLeft = d3.svg.axis()
                .scale(yScale)
                .ticks(3)
                .orient("left")
                .tickSize(-w, 0)
                .tickSubdivide(true);

'define’ an svg line for your chart

var line = d3.svg.line()
                .x(function(d) {
                    return xScale(d.date);
                })
                .y(function(d) {
                    return yScale(d.amount);
                })

at this point, you will be able to see NOTHING. no chart, zip, nada. you’ve only just declared variables. to start seeing stuff, we need to add this to our html, by selecting and appending.

'add’ an svg element to your page to represent the canvas

var graph = d3.select("body").select(".graph")
                .append("svg:svg")
                .attr("width", w)
                .attr("height", h);

'add’ and x-axis and a y-axis on top of the canvas

graph.append("svg:g")
                .attr("class", "x-axis")
                .call(xAxis)
                .selectAll("text")
                .style("text-anchor", "end")
                .attr("dx", 25)
                .attr("dy", h/2 + 11);

graph.append("svg:g")
                .attr("class", "y-axis")
                .call(yAxisLeft)
                .selectAll("text")
                .style("text-anchor", "start")
                .attr("dx", 10)
                .attr("dy", -5);

'add’ the line to the svg canvas on top of the axes

graph.append("svg:path")
                .datum(data)
                .attr("d", line);

'add’ little circles points to the canvas on top of the line

graph.selectAll("circle")
                .data(data)
                .enter()
                .append("circle")
                .attr("class", "data-points")
                .attr("cx", function(d) {
                    return xScale(d.date);
                })
                .attr("cy", function(d) {
                    return yScale(d.amount);
                })
                .attr("r", 5);

you can also add text, you can create tooltip popups, you can add transitions and transformations so the chart will animate itself based on events you define, like page load, onclick, onmouseover, etc, etc. there’s so many things!!!

i might do more posts in future as there are so many things it’s impossible to fit in just one post. stay tuned, or don’t, do whatever you wanna do…