Getting data is Painful

Actually downloading data where it has everything you want

As said last week, we got the first actual assignment. This is good and bad, it is good because we finally get something IT related rather than some research or reflection thing, however it is annoying because it means that I actually have to think more about doing the work. The assignment is basically what I was expecting (could be due to teacher telling us earlier), we have to get data, manipulate it (or just show it), and then reflect on it. My current issue is getting data that is similar between 2 places, and an actual archive of results.

The visualisation that I am doing is comparing Australia’s weather to US’s to determine if la Nina and El Niño means that they have the similar weather when on the same cycle. The issues that I am currency having is that it is hard to get useful historical data from BOM, and US weather requires an account to access (I just used 10min mail though), but more annoying thing is that they both have different ways of storing the data. I also used a very rudimentary way of determine the average temperature, averaging the min and max data (because not provided)

You can access my progress on GitHub (might need to look at branches). This is what I will be trying to (and at that link you can see the assignment task in full). I wonder if choosing a dataset that is harder is work, will require more time spent and possibly more things in reflection to write about (which is different to the blog posts, very confusing I know).


The most important thing that I have leaned is to try to visualize any data, this data doesn’t have to be the final one, but it gave me an understanding on what I needed to look for. This is why I started on a basic dataset and then went on to the weather, and then onto an even more advanced weather (where I actually got my question). I could have also done some more attempts, but I am happy so far (nothing is currently set in stone yet - maybe would be when closer to due date). Compared to last weeks I have put a bigger amount of effort, that is partly because it is an assessable item, but that it is actually interesting to do. As always there is lots things for improvement, this could be using class tome wisely, or spending time more usefully (doing the actual coding rather than spending time trying to find better data). The most frustrating part is the lack of consistency across government bureaus and how their respective data is laid out. I am happy enough for the data that I have currently gotten, even if it isn’t 100% what I wanted it to look like.

TL;DR: *Started assignment, hopefully good enough. Will do more of it the next week...*