Real-World Go with Referenced Functions

Writing small test scenarios and packages in the Go Playground, or in a single main.go file is all good and well, but in the real world things are more complex. (Or so I heard).

Today I wanted to get a brief sense of how that all works. I had some sense of how to make this all work when working through the GreaterCommons course, but it proved harder when the training wheels were taken away.

This is what I emerged with at the end of my experiments. It is a main.go file that references extended.go. This ended up being dependent on knowing a bunch of setup quirks and techniques that aren’t particularly well signposted for beginners. I ended up finding a tremendously helpful document on the main Go site entitled “How to Write Go Code”. They don’t signpost it much, especially when compared to ‘Effective Go’ or the general specification.

I worked my way through the official examples. Luckily they were simple enough that I could follow all of the logic. Everything was incrementally constructed, slowly becoming more complex. I learnt the following:

• it’s best to setup your workspace in $GOPATH/src/github.com/YOURUSERNAMEGOESHERE
• every time I initialise a GitHub repository on my computer (rather than creating it at GitHub.com and then simply cloning it down to my computer) I have to relearn all the little commands relevant to making that work when pushing up to Github. (It’s sufficiently fiddly and reminiscent of using PGP. Lots of small moving parts and room for error. Big space for improving that user experience…).
• if you use the path listed above as your workspace, you can reference your other folders fairly easily. See my code for examples of that.
• The packages you’re referencing (i.e. not in the central main.go file) can be called anything you’d like, as can the file names. If you want to make the functions in those packages globally visible, however, make sure the functions start with a capital letter. (In my example, I have extended.Extended).

I am already demonstrating some sloppy habits with my documentation, tests, benchmarks and examples, I realise. For these kinds of toy demonstration examples it is less mission-critical, but there feels like a vague principle at stake.

My next Go exploration project will probably be the final version of an Exercism exercise I’ve been working on for a while.

Table Tests in Go

Today I wanted to stretch my use of test scenarios with Go. The example I described a couple of days ago basically had me running individual tests for specific values. What I wanted was a way to test a bunch of different values for the same function. Enter: table tests.

You can see the code I ended up with partly in the image above but also on Github here. It took a while to get there.

I started with some notes I’d taken during Todd McLeod’s excellent GreaterCommons Go course. Those notes were enough to get a framework up and running. I understood the principle: you create a struct to store all the different values, loop over them all to check whether the test fails in any particular scenario.

When I ran go fmt at the end to format my code, it gave me an error as it refused to build:

View fullsize

I could see that it wanted two ints and I was giving it a slice of ints. Basically this turned into a hunt for fixing my loop and which values I was spitting out at various iterations of the loop.

I ended up isolating the part of the code that was causing the problems, putting it up on the Go Playground so as to isolate exactly what was going wrong. Once I’d figured out exactly how to handle the loop, I could then bring that logic back into my main_test.go file.

Now I know how implement table tests in Go. My next exploration will be around functions that aren’t located in the same file. So far I’ve been mainly using the same main.go file for all the code I’ve written, but a step up in the complexity will be to interact with different files.

Figuring out the Go testing ecosystem

I've been trying to figure out a way to work my way into increasingly complex Go explorations. I thought I'd start at the most simple and expand from there.

I thought I'd start off with a simple fmt.Println command. I loaded up GoLand only to find that I had 2 days left in my trial for the software. Open-source software is much more my scene, so I downloaded Atom and continued my coding there.

At first I wrote a basic draft of my code. I couldn't figure out how to test it, however, perhaps because my function wasn't returning anything, so I switched gears. No point getting distracted by these details. I wrote a function that definetely did return something: the sum of two integers.

[My code is all available here.]

I wrote examples for all my functions using the ExampleFunctionname syntax as described in the golang spec / documentation but I couldn't seem to get that to work. I'll have to return to that in due course.

Benchmarking worked fine. Go's benchmarking tools figure out how many times they need to run a particular function in order to properly test it. In my case, each function ran two billion times for an average of 0.29 and 0.3 nanoseconds per iteration. Pretty fast! I have no baseline for knowing how fast that is, but it seems fast to me. Of course, the function isn't doing anything particularly taxing.

My next task will be to figure out table testing so that instead of my current setup where I have the values for each test written out manually, and just one test per function, I want to have many different sets of values to test.

Machine Learning with Weka

Learning to program is an infinite process. The field is as open and wide as you can imagine, and you are mostly constrained by your imagination.

I spent much of May getting my mind around Go. I took Todd McLeod's Go course on Greater Commons and learned a great deal. The course was somewhat short on practical implementation, however, and I'm eager to do things with what I learned. More on that in due course.

A parallel strand of my studies has been in statistics and more advanced applications of statistical methods i.e. machine learning. I had done a bit of this in the past, but my poor foundation in basic statistics didn't serve me well. I am now rectifying that through Andy Field's excellent textbook.

For machine learning I decided to take a step back from the programming and use a graphical interface to start with. There are great APIs / tools available for this in most languages you can think of but I wanted more of a solid foundation in workflows around machine learning and the kinds of analysis that get done.

I read through a good deal of Jason Brownlee's blog(https://machinelearningmastery.com/) as well as his book on Weka and he made a good case for why Weka is a good place to start.

I have noted a number of steps to move through in sequence, at the same time recognising that data analysis is often unsequential. I expect this to expand and/or redefine this over time.

Kaggle is one of the major hubs for machine learning practice (and learning) and I wanted to reengage there. The first data set they usually have you work on comes from the Titanic disaster. You take the full roster of people who boarded, including data points like their economic class, where they were staying on board the ship and their age/gender etc and use anything and everything in terms of tools to predict who survived and who didn't. I had used this data set in the past when I was studying ML with Python.

My initial idea, therefore, was to take the .csv files from the Kagge competition and use them in Weka to come up with some predictions. Unfortunately, there are some idiosyncracies about the .csv file that make this difficult. Some of the attributes / columns in the data (like names) use punctuation marks which make parsing the csv data non-trivial. Weka uses ARFF files as standard but has the option to parse CSV data. It ran into quite a few errors when trying to crunch through the Titanic data, and no amount of basic fiddling would fix it.

Reading around a little, it seems that others have noted this problem in the past. One blog post tackled the problem head on but the solution didn't really help me much in the short term. I'm now somewhat stuck, knowing that the fix to the problem is to use another language (Python, perhaps) to range over the data and process it in a form that will be more palatable for Weka. Alternatively, I could use it as an opportunity to build a short Go programme that could perform the same function.

For the moment, i've decided to do neither. I'm going to find an alternative data set which doesn't require wrangling and fiddling. I know wrangling and fiddling is an important skill to master, but it's not the skill I'm trying to focus on right now. Luckily, between the UCI Machine Learning repository and various other places, I'm not exactly lacking for examples / other data sets. Today I'll work with the Pima Indians Diabetes data set which came built-in with Weka.