The JET project is about “creating an infrastructure for home monitoring and automation” (it’s actually considerably more, but this is a big-enough bone to chew on already…).
Note that JET is not about individual µCs or boards, it’s about managing an entire set of nodes, warts and all, and heterogenous from the start. JET is about bringing together (and bridging) lots of technologies, and about interfacing to existing ones as well. It’s also about evolution and long-term use - a decade or more - because redoing everything all the time is wasteful.
The JET infrastucture is not necessarily centralised, altough it will be in the first iterations: a “hub”, a variety of remote “nodes”, and browser access to use and administer it all. This is easy to map onto actual hardware, at least for a simple setup:
- the hub can be a Raspberry Pi or compatible, i.e. a Linux board
- the nodes will be JeeNodes, both AVR- and (in the future) ARM-based
- most communication will be wireless (sub-GHz, WiFi, whatever)
- most sensor nodes will be ultra low-power and battery-powered
- then again, control nodes could also run off USB chargers or similar
Nothing new so far, this story has not changed much in the past years, other than exploring different µC options and trying out some self-powered Micro Power Snitch ideas.
The hub is a recentintroduction: a portable application written in Go, in combination with the Mosquitto MQTT server. It has been running here at JeeLabs for a few months now, dutifully collecting home monitoring data, mostly from room nodes, the smart meter, and the solar inverter. The hub itself does very little, but it provides a way to add “Jet Packs” to run arbitrary processes which can tie into the system.
The browser side of JET has not changed: it’ll continue to be written in JavaScript (ES6) and will most likely use ReactJS and PureCSS as foundations. A lot of software development time will end up going there - this is not different from any other modern web-based development.
The nodes have all been running Arduino IDE based C/C++ code, most of this is available in JeeLib on GitHub, as far as ATmega- and ATtiny-based JeeNodes are concerned. Some newer experimental code for ARM has been presented in the past year on the weblog, some for LPC8xx µCs, but recently more for STM32 µCs. That code can be found in the Embello repository, see for example theRF Node Watcher.
But that’s where things are about to change - drastically!
A new beginning
From now on, new work on remote nodes will be done in Forth. Since Mecrisp Forth has proven itself to be very stable and flexible, it’ll be flashed onto every node - very much like a fancy boot loader. This is equivalent to making each node speak Forth on its serial port, once and for all.
This approach has been chosen, because Forth (in particular Mecrisp Forth):
- … is an interactive language
- … can compile code to flash on the fly
- … can clear (parts of) its flash memory again
- … can run “risky” code in RAM, with simply a reset to restore its previous state
- … could set up a watchdog to force such a reset, even on remote nodes
- … has very little overhead, the code is incredibly efficient
- … provides access to every feature available in hardware
- … can be configured to run arbitrary code after power-up or a reset
- … will fit in chips with as little as 32 KB flash, and just a few KB RAM
- … works on several different ARM families, not just the STM32 series chips
Do I have to learn Forth?
There are several possible answers to that question:
- if you only care about working code, the answer is “no” (just install firmware images)
- if you want to play with projects published on the weblog, the answer is “a little”
- if you want to dive in and explore everything, or change the code, the answer is “yes”
To explain that second answer: for trying out things written in Forth and made
available on GitHub, you don’t need to program in Forth, you can just enter
commands like “8686 42 rf-init”, “somevar @ .”, and such - easy stuff
(once properly documented and explained!).
Is everything going to be in Forth?
Nooooo! - Forth is still merely an implementation language for little µCs. The plan is to use it to implement a compact dataflow engine for remote nodes, which will then present a “gadgets and circuits” model, somewhat like NoFlo, Node-RED, and Pure Data. All data-driven.
Once such a basic dataflow engine exists, we will have a considerably more abstract conceptual framework to build with and on. There will be gadgets to tie into actual hardware (pins, digital & analog I/O, timers, but also the RF driver), and gadgets to perform generic tasks (periodic execution, filtering, arithmetic, range checking, conditional execution, etc). These can then be combined into larger circuits, and sent to a node as definition of the behaviour of that node.
The reason why Forth looks like a perfect fit for this task, is that it allowsgrowing a node’s functionality in small steps, once the Mecrisp core has been flashed into its flash memory. There will need to be a first layer to tie into the RFM69 radio modules (the RF69 driver already exists), and a way to robustly add and remove additional Forth code over the air. After that, we’ll need a dataflow core. Then, tons of “gadgets”, either coded in Forth or combined from existing ones.
At the end of the day/road/tunnel, Forth will end up being simply a low-level implementation language for manually coding only the bottom layers, with everything else generated from a visual diagram editor in the browser. The long-term goal is not to expose, but to bury Forth!
Yes, it will take a lot of work to get there - JET was never meant to be built in a day…