Visual programming and integration with Scratch, Node-Red and MQTT

Have had a little time to play and successfully managed to link Scratch to Node-Red as discussed in post “Visual coding and integration the requirement” The end result takes the pain out of connecting “things” from the outside world to and from Scratch.


Extensions for Scratch exist but tend to be specific to the thing they are connecting to. Two that I came across recently were for Leap Motion and CodeShield.  The idea behind using Node-Red is to create a system that makes it quick and simple to integrate many different types of “thing” with Scratch. The key being to do so with zero or the bare minimum of coding. Examples that spring to mind include mobile phone applications, weather stations, motion controllers, light and sound sensors and power monitors. In fact pretty much any type of device that you care to connect to boards like raspberry pi and beagle board or a standard computer.  In addition to devices Node-Red comes with a palette of nodes to integrate with social media like twitter, data stores and web services. The figure shows an example that integrates with a Node-Red instance on the same box as Scratch as well as integrating with a Raspberry Pi.

Scratch 2.0 is the newest version of Scratch and the one used in this post.  I get the feeling that I am trying to integrate Scratch with Node-Red at a time when the feature that enables Scratch to talk to the outside world is in flux. First the online web browser based version of Scratch had the “experimental extensions” feature removed just as I was about to experiment with it.  This led to the discovery of an offline downloadable version of Scratch which supports experimental extenstions.  Initial reading on how to integrate with it showed that communications between Scratch and the outside world could be established using a Socket or HTTP.  By the time I downloaded Scratch only HTTP was available. Given the amount of change that has occurred in a short period of time there is a reasonable chance that what I write here may not work as described in future versions.  I will try and keep abreast of and blog about changes as they occur. The article is based on:

Enabling Scratch 2.0 to talk to the outside world

This is achieved using the experimental extensions feature of Scratch. Documentation on it was a tad hard to find, the best source that I found was in this forum post in the Scratch support forums. Bare in mind if you looking at the post much of the early information is out of date. Scratch enables:

  • Data, events and commands to be sent from Scratch to the outside world using Command blocks
  • Data and events to be received into Scratch from the outside world using Report blocks.

The blocks to be used by the extension are configured in a JSON file. Here is a sample “extension.json” file:

   "extensionName": "Test",
   "extensionPort": 1880,
   "useHTTP": true, 
   "blockSpecs": [
     [" ", "test_Command1", "testCommand1"],
     [" ", "test_Command2 %n %n", "testCommand2", 100, 200],
     ["r", "test_Report", "testReport"]

Breaking out each line:

  • extensionName: what it says on the packet
  • extensionPort: the TCP port of the server that Scratch will connect to in order to talk to the outside world.  The server must run on the same box as Scratch as localhost is the only supported address.  In the sample 1880 is used which is the default HTTP port used by Node-Red.
  • useHTTP: somewhat useless now that only HTTP is supported.
  • Blockspecs: a set of Blocks that will be available for use in a Scratch project. Each spec is of the form blockType, display name, selector followed by optional parameters.  The selector is the name that will be passed to the outside world in order for the  extension server to understand what request is being made.
  • ” ” is a command block type that enables scratch to send commands, events or data to the outside world.
  • The first example with selector testCommand1 takes no parameters
  • The second example testCommand2 takes two parameters which will be passed to the extension server.  The parameters specified in the configuration file are the default values if not set by the Scratch application.
  • A control block of type “r” is a report block. This is Scratch’s way to ask the extension server for the value of the report block. In the example scratch will request the value of selector testReport.

The extension configuration is loaded into scratch by holding down the  shift key and selecting the File menu.  This will add a new Import Experimental Extension to the menu.  Select this and load the extensions JSON file. The blocks specified in the file will show up in the More Blocks set of script blocks.

How Scratch communicates to the extension server

Scratch is now configured to talk to the extension server. It uses the HTTP transport to provide the communications link.

For report blocks Scratch polls the extension server 30 times a second and expects the reply to contain the name and value of each report block.

  • Scratch polls the extension server appending a /poll to the url
  • The  extension server then responses passing back a set of name value pairs. One pair per line with the name and value separated by a space. The names must match the report block selectors in the JSON extension configuration file.

Having Scratch poll every 30 seconds is an inefficient mechanism for enabling communication. On my old computer Scratch consumes 20% CPU when not running an application.  I would like to see it move back to a socket or websocket approach providing bi-directional communication  between Scratch and the server.

For command blocks:

  • Scratch sends an http request appending the command selector name to the url. For instance /testCommand1
  • In cases where parameters are passed the parameters are added to the url separated by /. For instance /testCommand2/100/200

The image below shows a very simple Scratch application that invokes testCommand1 and 2 on the extension server and displays the value of the testReport sensor whenever it changes.


 Configuring Node-RED to understand Scratch

Node-Red provides a good starting point as it comes preconfigured with an HTTP server. This enables a web browser based application to be used to visually wire together nodes. As Scratch enables visual programming, Node-Red enables visual integration.  In addition to being used to create the wiring the HTTP server can also be used by applications such as Scratch that use HTTP to send requests to Node-Red.

The first Node-Red flow below shows how testCommand1 can be received and handled.


It is made up of a http input node, a function node followed by a http output node.  The input node is configured with a url of /testCommand1 which will be invoked when the testCommand1 control block is used in Scratch.  The example shown here is wired to a function block where code to handle the request can be placed.  This is just a sample of how it can be wired, Node-Red provides many other nodes that it could be wired to such as Twitter.  It is important that Scratch receives a http response in reply  to its request, the http output node makes this simple.

The next flow is very similar but shows how the command testCommand2 which needs to parse two parameters is handled.


Here the URL property of the input node is configured as:


param1 and param2 will be substituted with the value of the two parameters passed by Scratch. The parameters are then accessed in  the sample function block as follows:


Finally to the report block which provides the mechanism enabling data or events to be passed from the extension server to Scratch.  The figure below shows two flows:


The top flow demonstrates the use of the MQTT input node.  Here the mqtt protocol is used to connect to a MQTT server and subscribe to a topic that provides the wind direction in Winchester.  Whenever the wind direction changes the mqtt server will push an event to the mqtt input node.

Whenever the wind direction change event arrives, the function node is invoked. It is configured to store the wind direction in an object that is accessible to all flows / nodes. It does this by storing it in the global context space. For example: = msg.payload

The bottom flow consists of an http input node configured to handle requests on url of   /poll. Each time the poll request is received the function node is invoked. It sets the http response with the name and values of expected report blocks. In the case of the sample this is the testReport block. Here the sample sets the testReport block to the value of the wind direction:

msg.payload="testReport ";
return msg;

With the above flows deployed in Node-Red, the Scratch application can now send data,events or commands to the outside world as well as receive data or events from the outside world.

Using Scratch and Node-Red together looks like a powerful combination meeting the requirements outline in the last post including:

  • Connecting small devices in various forms to traditional computer systems and the creation of applications to visualise data from the devices and enable interaction with the devices.
  • Create demonstrations quickly yet ensure they look good.  
  • For teaching make it simple for novices to create new lessons/experiments involving devices and applications that can interact with them.

Small devices were not discussed directly in this article but given the remit of Node-Red to help wire the Internet of Things life should be good. Going forward there are a few extensions to be considered including:

  • Create custom Scratch input and output nodes in Node-Red.  Given the experimental extensions in Scratch are currently a moving target I’ll be waiting until the interfaces settle down.
  • Build on ideas in this post to create real world applications as opposed to some very simple samples that just show the principals of connecting Scratch to the outside world.

7 responses to “Visual programming and integration with Scratch, Node-Red and MQTT

  1. This looks to be an excellent extension. As mentioned on twitter my wife teaches primary school children and part of the new IT curriculum they have to show that they can read sensors and create basic programs. This looks to be a great way to use the friendly scratch interface to do this type of stuff and then hooking in to node-red to get the sensors all collected up

  2. Pingback: IoT / M2M Links #6 | Ian Skerrett

  3. Very cool! Can’t wait to use Scratch for IoT workshops (used PicoBoard before, but it’s too far away from real appications). Also, thanks for the details on how Scratch talks to extensions. Regards, Thomas

  4. Pingback: Mostly Armless + multiple sensor readings into Scrtach | davejlocke

  5. Pingback: Using Scratch & Scriptcraft (& more) to teach kids programming in Minecraft – blog MOOC

  6. Pingback: Using Scratch with Minecraft & Scriptcraft – step by step instructions | kids love to code

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s