code

Learning to Code with Robots

With STEM education being more prevalent these days, I was curious about a number of toys on the market geared towards teaching kids how to code. With all the options out there, which toy is the best investment? In the interest of scientific inquiry, I picked up four popular toys that support both simple block-based coding as well as advanced coding languages and gave them a try.

The Robots

Dash

Dash is an adorable little robot that was created by Wonder Workshop. It also has a little sibling, Dot, which is a non-mobile version of Dash. The two robots can be programmed to communicate with each other. The first thing you will notice about Dash is its giant white LED eye and the cheery “Hi!” greeting when you turn it on. Not only can the Dash move directionally, but it can turn its head and react to voices and claps. It also has one colored light on each side of its head and one colored light below its eye. As far as peripherals go, Dash has three embedded microphones for sensing sound, two infrared (IR) sensors for sensing distance, and a speaker to play sound. Dash can be programmed via an iPad or Android tablet.

Sphero

Sphero is the simplest robot of the group. It does one thing, but it does it well: roll. The entire ball lights up with RGB LEDs which can be controlled independently from the motion. Sphero is also surprisingly fast – it can reach a top speed of 4.5 miles per hour. There is also a neat clear version of Sphero aimed at education. The Sphero doesn’t have any external sensors per se, but it can detect impact and being picked up thanks to an internal gyroscope and accelerometer. When you first turn on Sphero, you have to do an orientation calibration routine so it can understand where you are in relation to the robot. Sphero can be programmed by an iOS (iPhone/iPad/iPod Touch) or Android device.

LEGO Mindstorm EV3

The most complex of the four robots is the LEGO Mindstorm EV3. This kit comes with a programmable brick, a handful of sensors, two motors, and 550+ LEGO Technic parts for creating just about anything you can imagine. To build a LEGO robot, you first build a LEGO structure then attach the programmable brick and various sensors or motors depending on what you want your robot to do. The sensors connect to the programmable brick via connector cables and the robot is programmed with a Mac or Windows computer. Although this method can be very time-consuming, it also seems to be the most flexible. There are many books and websites available to walk you through different robot builds and corresponding sample programs if you’re not quite sure where to start. For this evaluation, I built the standard TRACK3R robot from the Mindstorms manual using the infrared sensor for distance detection.

mBot

Another extensible robot is Makeblock’s mBot. Makeblock’s robots are built on top of an open source Arduino-based platform. The mBot is similar in spirit to the LEGO Mindstorm: you can combine a number of sensors with aluminum structure parts to come up with just about anything you can think of. Makeblock also offers many robotics kits with varying degrees of complexity, such as a 3D printer kit and a XY plotter kit (which can also be converted to a laser engraver). The mBot kit is specifically geared towards STEM education and comes with a number of sensors, such as an ultrasonic sensor, an infrared receiver, and a line follower as well as some on-board color lights. All robots on Makeblock’s platform can be programmed with a Windows or Mac computer using either their mBlock software or the Arduino software.

The Test Course

For the evaluation, I set up a simple evaluation course and assessed how hard it was to make the robot accurately navigate the course. The course consisted of these simple steps:

  • Go straight until it senses/runs into the first barrier
  • Flash the lights
  • Turn right
  • Go straight until it senses/runs into the second barrier
  • Flash the lights
  • Turn left
  • Go straight for a short distance
  • Spin in a circle three times
  • Flash the lights

Programming the Robots

For each robot, I used their block-based programming language to program the evaluation course instructions. For the uninitiated, block-based programming is a process where you drag block-like icons on a screen to create a chain of commands that represents a simple program. This method was widely popularized in education circles by MIT’s Scratch. By simplifying coding this way, people can become acquainted with the core concepts of programming without having to worry about the nuances of specific programming languages. Once someone is familiar with the basics of programming, it’s easier to understand more complex programming issues such as language syntax and scoping.

Dash

To program Dash, I used the accompanying iPad app called Blockly. The Blockly app has a several commands on the sidebar. To add a command to your program, simply click on the type of command you want to use, select the command you want and drag it over to the program area. The commands snap together to make a long vertical chain of commands, which are then executed when you click the start button. Blockly also supports using a few simple variables in the code if you want to keep track of things like the number of times Dash encountered an obstacle.

Blockly Program

One thing I really liked about Blockly was that many of the options were presented in terms of real-world values. So, for example, when you programmed Dash to move forward, you could select the distance in centimeters.

Dash Speed

All in all, the Blockly app was simple and easy to use. Connecting to the robot was as simple as holding down the robot icon until a green progress bar was full, indicating that the connection had been established. The only real issue I had with Blockly was that the app crashed on me a few times while trying to program Dash. This was not a serious deal breaker as my program was intact when I reopened the app.

Sphero

To program Sphero, I used the SPRK app on my iPad. Just like Dash, there are groups of commands at the bottom and you simply drag the command you want to use in the program and snap it into place. Once your program is ready, click the run button and Sphero will start executing the commands. The SPRK app allows you to modify preset variables such as speed and heading and well as create your own custom variables .

The SPRK app uses a different programming paradigm than the other robots. Instead of reacting to a single event, like having an obstacle in front, there is one block of code that is executed for every time a given event happens. This made programming Sphero a little abstract at times and could be hard for someone new to programming to understand.

Sphero On Collision

I didn’t particularly like the SPRK app that much. I found the large amount of unusable space on the right annoying considering that I could not rotate the app to use the space. I also found that the SPRK app did not give simple feedback if there was something wrong with a program. For example, I got this somewhat cryptic error when trying to flash the lights when a collision occurs:

SPRK Error

That being said, one thing I really did like about the SPRK app was that at any time you could click on a code icon in the upper left corner and see how your block-based code translated into their Oval coding language. Being able to look at the underlying code could be really useful when transitioning to writing code in the corresponding programming language. I was also pleased with how simple it was to connect the robot to the SPRK app. The app would automatically establish the connection after the initial Bluetooth setup.

Oval Code

Mindstorms EV3

I programmed the EV3 using the corresponding Mindstorms software on a Mac computer. With this software, blocks are laid out horizontally and can be broken into separate lines to help with readability. I connected my EV3 to my computer via a Bluetooth connection, which not only allowed me to program the robot remotely, but also allowed me to see the real-time sensor values in the lower right corner. Executing a program was as simple as clicking the download and run buttons in the lower right corner.

Mindstorms Software

One thing that is initially frustrating about the Mindstorms software is that, much like LEGO manuals, they don’t really use words anywhere. At times it’s not intuitive on how certain blocks should be used. Additionally, since the TRACK3R robot used tank treads, there was no simple “move forward” command, but rather I had to specify the power and direction of each tank tread. Movement is time-based, so there’s no simple way of translating a time into an actual real-world distance.

Even though the Mindstorms software feels a bit abstract at times, it is still pretty powerful. The software supports custom variables and also allows you to build custom blocks using the My Block Builder feature of the software. You can also add comments to your program to help you keep track of what you are doing. Additionally, the software allows you to add your own sounds and images to be used by the robot. One interesting feature of the EV3 is that the programmable brick has its own program editor so that you can modify the program on the robot without having to use the computer. As far as connecting the EV3, I found the Bluetooth connection to be a bit hard to establish at times. To fix this, I had to reconnect the EV3 to the computer via a USB cable just to reestablish the lost Bluetooth connection.

mBot

Makeblock has their own derivative of Scratch called mBlock. In fact, it still has many of the same elements as Scratch, so you can make a cartoon panda dance on your screen while your robot is moving about. I found this handy for understanding what the robot was doing at times – I could just have the panda display the sensor values on my computer screen while my robot was running. I programmed mBot using the mBlock software on a Mac computer, connecting to the robot using a 2.4 GHz wireless serial connection. Connecting to the robot was as simple as selecting the connection type I wanted to use in the Connect menu.

mBlock

The mBlock version of Scratch also has an Arduino mode, which allows you to see how your mBlock program translates to Arduino code. In order to use this mode, you cannot have any non-robot sprite commands in your program (so no dancing pandas). Much like Sphero, this helps you to visualize how the blocks translate to Arduino code. Unfortunately, the generated Arduino code can be a bit cryptic, especially for someone who may not be used to staring at written Arduino code.

mBlock Arduino

I thought that the mBlock software was really well designed and powerful. Those who have used Scratch before will find the software very easy to use. The window views are configurable so can you hide or resize different windows of the software. Like the Mindstorms software, mBlock allows you to create your own blocks or create custom variables for storing data. The mBlock app did crash on me a few times but I was easily able to reload my work from a saved file.

Evaluation

Here is a video of how each of the robots performed on the evaluation course:

Of the four robots, Dash was the fastest and easiest to program. The Blockly app was intuitive and Dash consistently executed the course as expected. The only issue I had with Dash is that it couldn’t navigate well on a rug.

The EV3 also made short work of the obstacle course. It took a bit of trial and error to figure out the exact motor settings for some of the tasks like turning. However, once the program was written, it consistently navigated the course without issue.

Sphero fared the worst of the four robots. Writing the Sphero code in an event-driven model felt somewhat unintuitive when compared to the procedural methods used by the other robots. Also, because Sphero really doesn’t have a front, the robot had to have the orientation calibrated each time I picked it up and reran the course. This quickly became annoying. Slight variances in calibration caused Sphero to veer off in different directions. It took multiple attempts to get the robot to execute the course correctly.

I really wanted to love mBot, but at the end of the day there were some issues with it. First, for some reason, the power to the wheels on my robot was not even, so my robot would always slightly veer to the right. A thorough inspection found no obvious reason for this and posts on the Makeblock forum showed that other people were experiencing the same issue with mBot. Second, the ultrasonic sensor readings were not normalized at all, so unexpected variances in the sensor readings sometimes caused the robot to prematurely turn. These issues made the evaluation runs far more frustrating than they should have been. Just like Dash, mBot also had issues navigating on a rug.

Conclusion

All in all, these are all great toys and any one of them would be an asset in getting anyone (especially kids) interested in programming. Dash was the easiest to use so I think it would be a great first robot for anyone, especially a younger child. The major drawback is that since most of the hardware was fixed, I could see this robot getting boring after a while. Furthermore, since Dash currently only supports programming languages with complex syntax like Objective C and Java, it would be harder to transition from the Blockly block-based programming to a full programming language.

I think the Mindstorms is the best option for people who want to have a platform on which they can grow. The LEGO hardware and software worked as expected without any issues. The Mindstorms software can be a bit confusing at first, but once you get past the initial learning curve, it’s very powerful in what it will allow you to do. As it’s been around the longest, it has lots of support material and has support for many programing languages, some of which are easier to learn (like Python). The major drawback to the EV3 robot is the high price point, which may not make it an ideal starter robot while you are still gauging your interest in coding.

The lower price point and great mBlock software still makes the Makeblock mBot kit an attractive option. My hope is that some of the initial kinks in the platform may later be worked out. It may be wiser to try a different Makeblock kit, like the slightly more expensive starter robot kit which comes with tank tread instead of wheels and a Bluetooth adapter which allows the robot to be manipulated through a mobile device. Much like Dash, the Makeblock robots can only be programmed with the complex Arduino language, which could make the transition to a full programming language more difficult. Fortunately, the Arduino mode in the mBlock software can help with that translation.

Comparison Chart

DashMindstorm EV3Sphero 2.0mBot
Price$170$350$130$79
Age Range5+10+8+8+
PowerRechargeable battery via micro USB6 AA Batteries, rechargeable battery pack (sold separately)Rechargeable battery via dock4 AA Batteries, rechargeable battery pack (sold separately)
Run timeAbout 5 hoursVaries on configurationAbout 1 hour Varies on configuration
ConnectivityConnectivity iOS (iPad) and Android via Bluetooth, Computer via USB (future) Computer via USB, Bluetooth, WiFi (adapter not included) iOS (iPhone/iPad/iPod Touch) and Android via Bluetooth, Computer via Bluetooth Computer via USB or Wireless Serial, WiFi (adapter not included), Bluetooth (adapter not included)
Beginner ProgrammingBlockly AppLEGO Mindstorms EV3 SoftwareSPRK App, Blockly Beta (via Chrome Browser), Macro Lab App, orbBasic App mBlock Software
Advanced ProgrammingObjective C, Java (both still in private alpha phase) Ada, C/C++, Python, Java, C#, Perl, VisualBasic, Lisp, Prolog, Haskell and more Objective C, Swift, Android, Python, Ruby, Arduino, Node/JavaScript and more Arduino
Included SensorsInfrared, microphone/sound Infrared (and tracking beacon), color/light, touch Internal gyroscope and accelerometer Infrared (and remote), ultrasonic, line follower
Optional SensorsNoneUltrasonic, sound, gyroscope NoneAccelerometer, compass, light, passive infrared, temperature, sound, touch
SoundsYes (fixed set)YesNoBuzzer only
LightsFront and side RGB lights, white eye light Red/Green/Amber LED on power brick One RGB light Two RGB LEDs on board, many RGB LED modules (sold separately)

Probabilistic Postal Address Elementalization

I haven’t posted in a few months because most of my time has been consumed with work and school. With respect to school, I have been taking Statistical Natural Language Processing at New York University. As my final project for this class, I have been working on something which I have been curious about for quite some time – probabilistic postal address elementalization.

What exactly does “postal address elementalization” mean? This is the process of breaking a postal address into tokens and classifying the function of each token, such as house number, street suffix or zip code. For my experiment, I created twelve distinct classes: five for administrative areas (country, state, county, city and neighborhood), one for postal codes and six for street address components (house number, prefix, pre-directional, base street name, post-directional and suffix). An example of an elementalized address is below:

address_elementalization

Although this seems like a straightforward problem, it is complicated by the fact that many countries have different languages and address formats. For example, in Ghana and Cameroon, there is no standard postal code system. In the Netherlands and Ireland, there is no province or state in the address. In some countries, such as France and Mexico, the postal code is placed before the city whereas other countries place it after the city. Furthermore, known names of terms in specific classes can overlap, so disambiguating streets can be tricky, like Avenue N in Brooklyn (“N” could be a post-directional or street name) or N Broadway in St. Louis (“St.” could be a suffix or part of a city name).

Traditionally, address elementalization has been implemented by building rule-based systems to handle each individual address format. This makes the process of building international address parsers very time consuming as one would need to implement a new rule-based parser for every single country. By building a statistical model to do this, implementing a new international format is as simple as training the model with a new data set.

Some interesting characteristics of postal addresses are that they have a grammar (as defined by the address format) and the elements are contextually dependent. These traits make the problem well-suited for natural language processing. There are natural language processing techniques that are used for similar purposes, namely part-of-speech taggers which are used to classify the parts of speech in a sentence.

For my final project, I looked at four different techniques of statistical part-of-speech tagging and applied them to the problem of postal address elementalization. The code is here.

The first strategy was a Hidden Markov model (HMM) tagger. HMMs are statistical models that can be used to find the most likely sequence of states for an input. This is done by using transition probabilities (the probability of a specific state given the previous state) and emission probabilities (the probability of the proposed state given the current input token). These probabilities are learned by observing a training set.  The model then tries to classify the input left to right, calculating the probability of a proposed state as the product of the transmission probability, emission probability, and maximum probability from the previous state. The model tries different state combinations over the input and returns the sequence of possible states with the highest overall probability.  A variation of this is the trigram HMM tagger, which uses the two previous probabilities to calculate the transition probability.

The second strategy was a Maximum-Entropy Markov model (MEMM) tagger. MEMMs are similar to HMMs in that they try to find the sequence of states that has the maximum total probability for an input. However, instead of just using observed counts for the transition and emission probabilities, we train a maximum-entropy distribution to calculate the probabilities. The main advantage of doing this is that we can have custom features factor into the probability, such as the current token length or whether the token contains a number. This allows for greater flexibility in tuning the tagger, but it makes the overall classification time slower.

The third strategy was a Transformation-Based Learning (TBL) tagger, also known as a Brill tagger. The TBL tagger generates rules based on observations in training. These rules indicate observed conditions on when a tag should be swapped with a different tag. During classification, initial tags are assigned to the terms based on the observed probability. The tagger iterates through the rules which were learned in training, swapping tags as necessary, until there are no more rules to be applied or a given score threshold is met.

The fourth strategy was a Conditional Random Field (CRF) tagger. CRFs implement a number of feature functions which take the proposed states and the input observation and return some value between 0 and 1. These features, like the features in MEMMs, can be just about anything, such as whether the current token is capitalized or whether it ends in -ed. Each of these feature functions has a different weight based on observations in training. The score for a sequence of states is calculated by summing the feature functions for every word over all words and then normalizing. Just like HMMs and MEMMs, we find the sequence that maximizes the overall probability.

From my initial work, I found that the Maximum-Entropy Markov model and the Conditional Random Field taggers consistently had the highest overall accuracy of the group. Both consistently had accuracies over 98%, even on partial addresses. For full format American addresses, these taggers had an accuracy around 99.7%. If I had to choose between them, I would go with the Conditional Random Field tagger as it was considerably faster in tagging the addresses.

I didn’t have enough time to finish everything that I wanted to implement, so this is still a work in progress. I still want to implement some smoothing techniques for unknown states, such as Katz backoff and Kneser-Ney interpolation. There are also a few more part-of-speech tagging techniques I would like to experiment with. I guess that old adage is true in this case – time flies when you are having fun…  🙂

Arduino and Sensors

I’ve been busy putting the finishing touches on a class I am teaching at NYC Resistor called Arduino and Sensors. The purpose of this class is to teach people how to use common sensors with Arduino so they can build awesome interactive projects. The class features the Adafruit Sensor Pack 900, as this pack contains a nice selection of common sensors. I’ve written some sample code for each of the sensors in the pack. We will discuss both digital signal and analog signal sensors.

Sensors!

Digital signal sensors are the simplest to use. They simply return a 1 or a 0 based on the reading of the sensor (just like a switch, it’s on or off). Therefore, reading the state of one of these sensors is as simple as hooking the output of the sensor to a digital pin on the Arduino (pins 2-13 on the Uno) and calling digitalRead() on that pin. Here is a simple example – an IR sensor:

IMG_6848

Analog signal sensors are more complex. These sensors return a voltage on an analog pin somewhere from 0 volts to the max voltage of the microcontroller (with the Uno, it’s 5 volts). In order to read an analog sensor value, the sensor output needs to be connected to an analog pin on the Arduino (pins A0-A5 on the Uno). In the code, calling analogRead() on the analog pin will give you the sensor reading. The Arduino automatically converts the voltage on the analog pin to an integer between 0 (no power) and 1023 (full power). Generally, the reading can be mapped back to some meaningful value. For example, here is a simple analog sensor – a temperature sensor:

IMG_6832

According to the datasheet, this sensor returns 0 volts at -50 degrees Celsius and 1.75 volts at 125 degrees Celsius. It has a scale of 10 millivolts per degree Celsius. To get the raw voltage reading, we take our reading value, divide it by 1024 (to get the percentage of the full voltage) and then multiply that by 5 (since the microcontroller is supplying 5 volts). To scale the voltage to the range, we can simply multiply the voltage by 100 (according to our scale factor, 1/100 volt is 1 degree Celsius) and then subtract 50 (since zero volts is -50 degrees Celsius).

Of course, many sensors are more complex than just reading a simple pin. We’ll discuss a number of different scenarios and how to handle them. Did you ever wonder what the AREF pin is for? It’s the analog voltage reference pin and we will be discussing how to use it. We’re also going to use potentiometers to tune the sensitivity of some of the sensors.

The class is already almost sold out! If all goes well, I will hopefully teach it again soon!