Tag Archives: visual integration

Visual programming and integration with Scratch, Node-Red and MQTT

Have had a little time to play and successfully managed to link Scratch to Node-Red as discussed in post “Visual coding and integration the requirement” The end result takes the pain out of connecting “things” from the outside world to and from Scratch.

simplescratchapp

Extensions for Scratch exist but tend to be specific to the thing they are connecting to. Two that I came across recently were for Leap Motion and CodeShield.  The idea behind using Node-Red is to create a system that makes it quick and simple to integrate many different types of “thing” with Scratch. The key being to do so with zero or the bare minimum of coding. Examples that spring to mind include mobile phone applications, weather stations, motion controllers, light and sound sensors and power monitors. In fact pretty much any type of device that you care to connect to boards like raspberry pi and beagle board or a standard computer.  In addition to devices Node-Red comes with a palette of nodes to integrate with social media like twitter, data stores and web services. The figure shows an example that integrates with a Node-Red instance on the same box as Scratch as well as integrating with a Raspberry Pi.

Scratch 2.0 is the newest version of Scratch and the one used in this post.  I get the feeling that I am trying to integrate Scratch with Node-Red at a time when the feature that enables Scratch to talk to the outside world is in flux. First the online web browser based version of Scratch had the “experimental extensions” feature removed just as I was about to experiment with it.  This led to the discovery of an offline downloadable version of Scratch which supports experimental extenstions.  Initial reading on how to integrate with it showed that communications between Scratch and the outside world could be established using a Socket or HTTP.  By the time I downloaded Scratch only HTTP was available. Given the amount of change that has occurred in a short period of time there is a reasonable chance that what I write here may not work as described in future versions.  I will try and keep abreast of and blog about changes as they occur. The article is based on:

Enabling Scratch 2.0 to talk to the outside world

This is achieved using the experimental extensions feature of Scratch. Documentation on it was a tad hard to find, the best source that I found was in this forum post in the Scratch support forums. Bare in mind if you looking at the post much of the early information is out of date. Scratch enables:

  • Data, events and commands to be sent from Scratch to the outside world using Command blocks
  • Data and events to be received into Scratch from the outside world using Report blocks.

The blocks to be used by the extension are configured in a JSON file. Here is a sample “extension.json” file:

{
   "extensionName": "Test",
   "extensionPort": 1880,
   "useHTTP": true, 
   "blockSpecs": [
     [" ", "test_Command1", "testCommand1"],
     [" ", "test_Command2 %n %n", "testCommand2", 100, 200],
     ["r", "test_Report", "testReport"]
   ]
}

Breaking out each line:

  • extensionName: what it says on the packet
  • extensionPort: the TCP port of the server that Scratch will connect to in order to talk to the outside world.  The server must run on the same box as Scratch as localhost is the only supported address.  In the sample 1880 is used which is the default HTTP port used by Node-Red.
  • useHTTP: somewhat useless now that only HTTP is supported.
  • Blockspecs: a set of Blocks that will be available for use in a Scratch project. Each spec is of the form blockType, display name, selector followed by optional parameters.  The selector is the name that will be passed to the outside world in order for the  extension server to understand what request is being made.
  • ” ” is a command block type that enables scratch to send commands, events or data to the outside world.
  • The first example with selector testCommand1 takes no parameters
  • The second example testCommand2 takes two parameters which will be passed to the extension server.  The parameters specified in the configuration file are the default values if not set by the Scratch application.
  • A control block of type “r” is a report block. This is Scratch’s way to ask the extension server for the value of the report block. In the example scratch will request the value of selector testReport.

The extension configuration is loaded into scratch by holding down the  shift key and selecting the File menu.  This will add a new Import Experimental Extension to the menu.  Select this and load the extensions JSON file. The blocks specified in the file will show up in the More Blocks set of script blocks.

How Scratch communicates to the extension server

Scratch is now configured to talk to the extension server. It uses the HTTP transport to provide the communications link.

For report blocks Scratch polls the extension server 30 times a second and expects the reply to contain the name and value of each report block.

  • Scratch polls the extension server appending a /poll to the url
  • The  extension server then responses passing back a set of name value pairs. One pair per line with the name and value separated by a space. The names must match the report block selectors in the JSON extension configuration file.

Having Scratch poll every 30 seconds is an inefficient mechanism for enabling communication. On my old computer Scratch consumes 20% CPU when not running an application.  I would like to see it move back to a socket or websocket approach providing bi-directional communication  between Scratch and the server.

For command blocks:

  • Scratch sends an http request appending the command selector name to the url. For instance /testCommand1
  • In cases where parameters are passed the parameters are added to the url separated by /. For instance /testCommand2/100/200

The image below shows a very simple Scratch application that invokes testCommand1 and 2 on the extension server and displays the value of the testReport sensor whenever it changes.

simplescratchapp

 Configuring Node-RED to understand Scratch

Node-Red provides a good starting point as it comes preconfigured with an HTTP server. This enables a web browser based application to be used to visually wire together nodes. As Scratch enables visual programming, Node-Red enables visual integration.  In addition to being used to create the wiring the HTTP server can also be used by applications such as Scratch that use HTTP to send requests to Node-Red.

The first Node-Red flow below shows how testCommand1 can be received and handled.

flow1

It is made up of a http input node, a function node followed by a http output node.  The input node is configured with a url of /testCommand1 which will be invoked when the testCommand1 control block is used in Scratch.  The example shown here is wired to a function block where code to handle the request can be placed.  This is just a sample of how it can be wired, Node-Red provides many other nodes that it could be wired to such as Twitter.  It is important that Scratch receives a http response in reply  to its request, the http output node makes this simple.

The next flow is very similar but shows how the command testCommand2 which needs to parse two parameters is handled.

flow2

Here the URL property of the input node is configured as:

/testCommand2/:param1/:param2

param1 and param2 will be substituted with the value of the two parameters passed by Scratch. The parameters are then accessed in  the sample function block as follows:

msg.req.params.param1
msg.req.params.param2

Finally to the report block which provides the mechanism enabling data or events to be passed from the extension server to Scratch.  The figure below shows two flows:

flow3

The top flow demonstrates the use of the MQTT input node.  Here the mqtt protocol is used to connect to a MQTT server and subscribe to a topic that provides the wind direction in Winchester.  Whenever the wind direction changes the mqtt server will push an event to the mqtt input node.

Whenever the wind direction change event arrives, the function node is invoked. It is configured to store the wind direction in an object that is accessible to all flows / nodes. It does this by storing it in the global context space. For example:

context.global.winddirection = msg.payload

The bottom flow consists of an http input node configured to handle requests on url of   /poll. Each time the poll request is received the function node is invoked. It sets the http response with the name and values of expected report blocks. In the case of the sample this is the testReport block. Here the sample sets the testReport block to the value of the wind direction:

msg.payload="testReport "+context.global.winddirection;
msg.statusCode=200;
return msg;

With the above flows deployed in Node-Red, the Scratch application can now send data,events or commands to the outside world as well as receive data or events from the outside world.

Using Scratch and Node-Red together looks like a powerful combination meeting the requirements outline in the last post including:

  • Connecting small devices in various forms to traditional computer systems and the creation of applications to visualise data from the devices and enable interaction with the devices.
  • Create demonstrations quickly yet ensure they look good.  
  • For teaching make it simple for novices to create new lessons/experiments involving devices and applications that can interact with them.

Small devices were not discussed directly in this article but given the remit of Node-Red to help wire the Internet of Things life should be good. Going forward there are a few extensions to be considered including:

  • Create custom Scratch input and output nodes in Node-Red.  Given the experimental extensions in Scratch are currently a moving target I’ll be waiting until the interfaces settle down.
  • Build on ideas in this post to create real world applications as opposed to some very simple samples that just show the principals of connecting Scratch to the outside world.
Advertisements
Aside

A number of projects have come across my path recently that led to this blog post and I suspect a few more down the line. One project involved the creation of several one off demonstrations and the other a project … Continue reading