Friday 30 June 2017

Distributed Monitoring Projects - RIPE ATLAS & FlightAware FlightFeeder

I'm currently hosting nodes for two distributed monitoring networks - one for several years now (since perhaps 2010 or so), and the other one as of yesterday.

Distributed monitoring networks put small, low power, low bandwidth devices into your network to get a better view of various things of global scope.

The two I'm currently involved in are RIPE Atlas Probes and FlightAware's FlightFeeder.



RIPE Atlas Probes


RIPE Atlas Probes are small devices which allow anyone to conduct distributed monitoring of networks and networked devices across the globe (over 10,000 devices scattered across the Internet). I host two v1 probes (which shows you how long I've been doing this!), one on my home ADSL network, and one at the previous organisation I worked with - I've left the latter there, as it has native IPv6, and our school currently does not.

RIPE's Atlas's site has numerous resources that tell you more about the project, and of course the Probes themselves. If you're interested, go to the Get Involved page. If your location is interesting to RIPE, they may well ship you a free probe if you sign up.

If you're a (much) more advanced user with a chunk of change you need to get rid of, RIPE Atlas also makes use of Anchors, which are much more fully featured and more powerful - but you'll have to cover the costs yourself, and upgrade the hardware regularly. Generally, they need to be in fairly "reliable" locations (so if you have a highly available datacentre, it's interesting, but a home ADSL network probably wouldn't be).

One of the "useful" functions to the broader Internet community of a network like this is that it enables a near-realtime view of reachability of critical infrastructure like the root DNS nameservers.

I've also noticed that companies with a big internet presence use them to see if their customer base is likely to be able to reach their websites - locally, I've noticed banks doing this (e.g.); below is a list of every target with the local ccTLD on one of my probes:
It looks like someone added quite a few very close in time to each other...
If you look at the measurement numbers, a lot of them are very close in number and time started - this suggests some enterprising individual has sold a measurement service to a bunch of organisations in the country!

Other than the "egoboo" of hosting useful infrastructure, you can get something out of the project - you can load your own measurements onto any probe(s) you like; hosting a probe gets you free measurement credits (which you might otherwise buy), so you can monitor various hosts remotely, and do various other measurements.

RIPE Atlas Probes don't use much power (typically USB powered) or bandwidth and are cheap (i.e. free). If you belong to an autonomous system (or region; most countries in Africa have none, for example) that doesn't host an Atlas Probe or it poorly represented, consider signing up today! If you're likely to come across large gatherings of people that would be interested in hosting a probe (particularly in under served regions) have a look at the Ambassador programme.

FlightAware's FlightFeeder

What planes are flying around us?
Skyview webpage from our FlightFeeder

FlightFeeder is a customised Raspberry Pi in a box which monitors aircraft, mostly through a protocol called ADS-B (it also receives other Mode S transmissions). They ask you not to open the thing as a condition of use, so I've not peeked inside, but essentially, they're Raspberry Pi boards with a digital radio, giving them accurate time and location (through GPS) and being able to receive ADS-B transmissions from aircraft - and of course, some custom software to make the magic happen.

It's all green and working!

The rise of Software-Defined Radios (SDR) has meant that it's becoming increasingly cost-effective to create interesting applications on basic platforms like RPi to do interesting things.

Of course, being built on a Raspberry Pi, you're correct in thinking this programme is open to people who like to build things. A version of their software (PiAware) is made available for those who would like to tinker. Of course, as well as a Raspberry Pi (v3) you'll also need a receiver (a USB SDR) and appropriate antenna that can pick up the ADS-B frequency (most commonly 1090MHz). Building them is quite straightforward, and if you have access to Amazon.com, it's easy to source the requisite components; the antennas and radios can be the hardest bits to find. Antennas also make interesting projects for inveterate tinkerers (for example this or this).

Of course, the closer you live to a (busy) airport, the more interesting it gets. If you don't like Pi, then you could (in Windows) look at things like Virtual Radar Server - but you'd need a device that feeds it data (like this or this). I don't think they'd be nearly as fun as building on a Pi though. Just learning how to format the SD card and install the OS is a useful life skill!

If you'd like to see if your site is acceptable for a FlightFeeder (anyone can volunteer PiAware data), there is a basic sign-up process. You'll need photos of N/S/E/W views from where you want to mount it and a picture of intended mount location as part of the application if you pass the first step. Of course, anyone is free to contribute. Their website lists areas that they're particularly interested in covering. As with many radio things, it's a line-of-sight protocol, but as many aircraft fly rather high up, line of sight can be quite far away. FlightRadar24 also offers free gear to selected sites. I suspect given their partnership with Aireon, who run ADS-B receivers on Iridium satellites, giving global coverage, FlightAware are likely to leave the other services "in the dust", so to speak.

One technique that comes out of reasonably dense deployments of this sort of thing is Multilateration (MLAT). With accurate timing data (GPS or perhaps NTP) and careful synchronisation, you can locate things quite accurately, particularly once an object can be "heard" by three or more independent stations at the same time. To learn more, have a look at this site; this is what FlightAware does with MLAT. It's a little bit like GPS in reverse - you know where you are, but you're trying to figure out where something else is - Time of Arrival is critical. If you've ever done triangulation with a map and compass, it's more or less similar, too. If you've ever wondered why GPS gives you accurate clock data, this is why - precise timing of the arrival of signals from various satellites in known orbits allows you to calculate your position. Reverse that concept from three or more known locations and you can pin-point the location of an object by comparing the time when the signal arrives. The more accurate the clocks (and their synchronisation) the more accurate the location will be. Of course, ADS-B gives GPS-derived location, heading and speed, but having an additional way of verifying location is important (in case an aircraft "lies" or some nefarious individual injects rubbish data) - and of course MLAT can work out where anything transmitting identifiable data on 1090MHz might be, even if it's not transmitting ADS-B signals.
Typically, you'll want at least 10 kilometers or so between radios if you want to ensure that you have more or less "guaranteed" MLAT near you. If you have friends with big towers, or even homes, who wouldn't mind a few milliamps of power and kilobits of Internet, you might want to investigate deploying some of your own; it wouldn't be too hard to assemble suitable weatherproof housings. The Flight Crew features might let you create a "federation" of users who pool data, rather than registering all the devices to one account; you can see the power of this in the screenshot below.
Top ten "flight crews" submitting data to FlightAware.
It's interesting to note that this stuff is powerful enough that when deployed professionally (on platforms that probably cost a LOT more than an SDR and a Raspberry Pi...) they can replace (and greatly outperform) air traffic control RADAR (Secondary Surveillance Radar). Properly employed, it is likely to improve air traffic control efficiency, and even reduce carbon emissions by more efficient routing and handling of aircraft by ATC. Some areas are already making extensive use of it, and it is likely that most larger aircraft will soon be fitted with it.

Of course, aircraft themselves can also receive the data, and help them ensure they don't collide with other planes (so-called "situational awareness"). You can even get it for drones!

Obviously, they also keep a LOT of statistics about your deployment; it's quite interesting to see what sort of things you're picking up and in what directions. I still have to finish installing the antenna in the proper location, but I'm already picking up aircraft hundreds of kilometers away from a somewhat sub-optimal location. You can view anyone's stats online (although this may be members-only)
Coverage graphs, showing direction and distance of detected aircraft.
If you get very into this sort of thing, a more capable (and of course more expensive) BeagleBone based platform is Radarcape. If you'd like to do a little soldering, there is also the Mode-S Beast kit - which will also take more work to get something useful out of. Some Googling will likely turn up more interesting things, if this piques your interest. Having several radios in reasonably scattered locations will help with increasing the coverage by MLAT (and generally how much of the sky you can "see").

There are a number of other sites that aggregate user/community submitted ADS-B data; FlightAware is particularly nice because they'll send you what amounts to a fair chunk of kit for free in return for the data you can supply (and even give you more access to some premium site features for free).

Tracking Ships

In a "former life", I spent a lot of time at sea aboard research vessels. One of my favourite bits of technology was AIS, particularly once integrated into a chartplotter (one vessel I was on in 2008 even integrated multibeam swath bathymetry into their chartplotter as they travelled). AIS is similar to ADS-B, but provides even more information. It works at marine VHF frequencies. Of course, again, being a unencrypted file format, people make receivers and decoders for that, and there are services that will possibly send you one for free (see also pocketmariner). Again, you can probably make one with a Pi, but it's not quite as easy as with ADS-B. If you live near to the coast - and particularly, near to a busy port, it may be quite interesting to have one; we're unfortunately a bit too far inland (and live in a bowl) to make this interesting (LoS is around 70km before the earth gets in the way, even without hills).

What use is this to schools?


Although I'm not a teacher, I find that having random "geeky" gadgets around can foster interests in kids that are often not well catered for through the normal curricular and extra-curricular activities offered by a school. Whilst it can be hard to find time when you have a full-time sysadmin job, I think it's part of your "implicit" duties to try and carve out at least the odd slice of time to interact with kids on a meaningful level.

Little projects of this type can be interesting to kids - if you're a well resourced school, you might be able to have students build such things (perhaps in a makerspace or something similar); if you're less well resourced, free is good, and exposure to interesting gadgets can help spark interest, creativity and innovation in pupils.

When you find (for example) people who are obsessed with planes, having what amounts to a live RADAR view of the skies above is pretty neat. It's fascinating, in particular, to find smaller planes that you didn't expect there; just this morning, I came across a Gulfstream V that must belong to the phenomenally wealthy owners of a nearby game reserve (which has its own high end landing strip), another private charter jet, a Beech F90 from 1983 that was exported to Botswana from the USA, and what must have been an IFR-equipped Piper Cub from an air school 60-odd kilometers away - as well as plenty of scheduled flights in and out of the closest major airports.

If you can do so safely, it's particularly rewarding for pupils to get involved in installation and maintenance (and troubleshooting!).

Such networks start to touch on the potential of the so-called "Internet of Things"; development of applications leveraging that is likely to be a definite career path.  Similarly, you have distributed processing projects, like Seti@HomeBOINC or Folding At Home (or blockchains...!)

I think the trick is to frame things as questions that get pupils thinking about problems - and then getting them to implement either their solutions (where feasible) - or something else. I had several teachers who taught more or less this way (i.e. asking about your hypothesis about why a thing was, or what it might be like, before expounding on the "accepted" answer) - very effective in my experience. When I was at school in the 1990s, I helped to install the first networking in an Acorn-based computer lab (10base2, ick) and a satellite weather system in the geography department; we also played with the first digital cameras and basic handheld scanners. This sort of thing sticks with you!

Many kids enjoy competition - "who can make the best antenna?" for purpose X is a great competition, and might end up enriching them with some radio physics in the process (a handy curriculum link!). Of course, if radio physics is your thing, engaging in "ham radio" will really get to the heart of the matter.

Knowing "how to think" is a surprisingly useful skill that is poorly taught, even today - when "teaching for the test" still really just gets people (in most subjects) to be good at (often temporarily) memorising some things, spewing them out on paper, and moving on with life. This is quite different to the skills that you will actually need in most workplaces (excelling at quizzes is rarely a lucrative career) - with Google a moment away, memorising/"parroting" things is less and less useful/relevant. People can be a little put out by being asked to think in a classroom - but that "aha" moment when someone finally "gets it" is so worth it! It can also help you gauge what people "already know" - and what they still know/understand some time later, and it might help you to get people to think about problems in different ways (which is also often a good way of developing troubleshooting/problem solving skills). Physical artifacts can go a long way in enriching this learning/discovery process, no matter how much you think "learning styles" are real or not.

Feeding off this type of experience, I recently had a scholarship kid from a disadvantaged background (i.e. someone unlikely to be able to afford their own computer) taking IT as a subject build his own PC from obsolete parts we were ultimately scrapping, and install Ubuntu on it, which is perfect for him to get on with IT schoolwork outside of classroom time. I suspect he's learnt a lot more than he realises from doing this, and it brings to life a lot of stale concepts from theory, particularly if you know bits of the curriculum and can sneakily embed them into Q&A about what they're doing, and why - and what the bits are - and do! Of course, Ubuntu is also an amazing platform for tinkering! Whilst I'm not sure most school administrators would enjoy putting valuable equipment in the hands of kids, it would be quite fun to make your IT classes build a PC before they can even do IT!

No comments:

Post a Comment