Telemetry and Video over 4G with Ardupilot

I spent two months in South Africa with WILD, a non-profit organization that develops innovative solutions for wildlife conservation. The project I worked on is the Air Ranger, a big fixed-wing UAV that monitors the border of the Dinokeng Game Reserve. The goal was to improve the range of the video-transmission system and telemetry. The 2.4GHz pollution in that area prevented us from using more traditional systems. Even the DJI drones in FCC mode have very limited range in some areas of the park.

Therefore I opted to try out a solution based on 4G, since the cell-tower coverage of the park seemed to be very good. In this blog post, I describe a step-by-step approach to setting up 4G for a Mavlink based autopilot system. While routing Mavlink over 4G is certainly not something new, I went two steps further by also implementing an adaptive bitrate algorithm for the video, which greatly improves usability and reliability of the telemetry feed. I also developed a script that maps signal quality parameters to 3D coordinates, which helps to understand the coverage and signal quality for the area.

These methods have been tested successfully up to a distance of 16.2km. That was the furthest point in the park relative to our takeoff location.

The Air Ranger

The Air Ranger is designed specifically with anti-poaching operations in mind. Its main features are the low cruise speed and low noise profile. Despite the weight, it is more quiet than your average multi-rotor. The low cruise speed makes it easier to monitor the video from the thermal camera. Even with a thermal camera, it can be difficult to recognize people. Flying slower gives you more time to observe. Another ongoing project is to use AI, which would allow us to fly faster.

Some of the specs of the airplane are:

• 3 meter wingspan
• 8 kg takeoff weight
• 9-10 m/s cruise speed
• 2.5 hours flight time
• 6S6P Li-ion battery (18650 cells) capacity 31.500 mAh

Raspberry Pi and 4G router

A Raspberry Pi (RPi) is perfectly suitable for converting Mavlink serial telemetry to UDP and is also capable of transmitting live video. I chose a Raspberry Pi 3B+ since that was available, but any decent single board computer should do. I have previously used a RPi Zero 2W for this purpose, which is nice to use in smaller airplanes.

For the 4G connection, there are plenty of options too. Initially I used a Huawei E3372 USB dongle, but later I changed that for a Huawei E5783 MiFi router. I've even done a flight with our 'home'-router, a Huawei 4G CPE 3. It doesn't really matter, but pay attention to bitrate. They come in variations of 50Mbps, 150Mbps and even 300Mbps. While you will never reach those numbers in practice, a higher number is still going to be better.

There seems to be a correlation between modem temperature and losing connection. Especially when flying further away from cell towers, the modem needs to send a more powerful signal. This heats up the modem itself and on various occasions that would drop the connection entirely, only to come back after several minutes.

When I placed the modem in the airflow outside the fuselage, I would never experience any problems like that. So while I don't have the data to back this up, I have seen enough to conclude that it needs some form of cooling.

Tailscale VPN

When you have the RPi connected to the internet via a 4G modem, you still need a way to connect to the RPi. Most 4G providers will not give you a public IP address. You really don't want this either, because that allows everyone with your IP to send random stuff and eat up your data.

There are several ways to open a connection to your RPi. I recommend Tailscale to begin with, which is the easiest to get started and usually just works. Create an account, install Tailscale on both your computer and the RPi. You can then connect directly to the RPi via its Tailscale IP.

Other options are:

• Wireguard
• OpenVPN
• Reverse SSH tunnel
• ZeroTier
• Private APN with your 4G provider
• And many more

While Tailscale works perfectly for an initial setup, it doesn't scale out to teams unless you want to pay for their service. For this reason, I switched over to a self-hosted Wireguard server. This is still relatively easy to setup using wg-easy. You will need a server with a public IP address for this. I had some issues with Wireguard where some packets got dropped, but this was resolved by reducing the MTU to 1280.

Routing Telemetry with Mavproxy

To get Mavlink to the RPi, we need a connect a cable from the pixhawk TELEM1 or TELEM2 port. You can also use other ports as long as you configure it to output Mavlink. On the RPi, you can plug the RX and TX pins directly onto the pin headers. Check pinout.xyz to find which pins are UART TX and RX. If you use these pins, you also need to run raspi-config and configure the serial port to be enabled, but not attach a serial console. Your serial device will be /dev/ttyS0

Alternatively you can use any USB-to-UART device, for example a CP2102 or FTDI dongle. If you don't have other dongles attached, your serial device will be /dev/ttyUSB0.

Install Mavproxy on the RPi by following the installation instructions. Personally I like to use a python virtual environment for this, but that's up to you.

To test the connection, open QGroundControl on your computer and run the following command on the RPi:

mavproxy.py --source-system=0 --master=</dev/ttyS0 OR /dev/ttyUSB0> --out=udp:<COMPUTER TAILSCALE IP>:14550

It is very important to set --source-sytem=0 here. Otherwise, the Mavproxy tool itself will emit a heartbeat which will be recognized by Ardupilot. Therefore it will not trigger any failsafe when the 4G connection gets lost.

If this works, you want this command to start automatically when the RPi boots up. This can be achieved by creating a system service. Create the following file at /etc/systemd/system/mavproxy.service:

[Unit]
Description=MAVProxy Service
After=network.target

[Service]
ExecStart=mavproxy.py --source-system=0 --master=</dev/ttyS0 OR /dev/ttyUSB0> --out=udp:<COMPUTER TAILSCALE IP>:14550
Restart=always
User=pi
WorkingDirectory=/home/pi
StandardOutput=append:/var/log/mavproxy.log
StandardError=append:/var/log/mavproxy.log

[Install]
WantedBy=multi-user.target

Then start the service and enable it to start automatically on next boot:

sudo systemctl daemon-reexec
sudo systemctl enable mavproxy.service
sudo systemctl start mavproxy.service

Low-latency video with MediaMTX

In order to transmit video with low-latency, I recommend MediaMTX. There is a separate repository for MediaMTX with rpi-camera. You can install this on the RPi directly, but I prefer to run this in a docker container. For that, I use docker compose and the following docker-compose.yml:

services:
  mediamtx:
    image: bluenviron/mediamtx:latest-rpi
    container_name: mediamtx
    privileged: true
    network_mode: host
    volumes:
      - /run/udev:/run/udev:ro
      - ./recordings:/recordings
      - ./config:/config
    tmpfs:
      - /dev/shm:exec
    command: "/config/mediamtx.yml"
    restart: unless-stopped

For this to work, you need to place a mediamtx.yml config file in the config directory. You can use the base config from GitHub as a starting point.

If you use a RPi-camera, set this at the bottom of the file. This will make your camera feed available at http://<RPI IP>:8889/rpicam.

paths:
  rpicam:
    source: rpiCamera

Increase reliability with adaptive video bitrate

A big issue with video over 4G is potential bottlenecks with the 4G connection. When packets queue up in the 4G modem, it adds latency and packets get dropped occasionally. This is quite bad when trying to operate a UAV in real-time. To make matters worse, the SSH connection is also affected so you can't easily shut down the video stream.

The simplest work-around for this is to use a very low bitrate. By limiting the video stream to only 500 kbps, the chance of hitting the limit is a lot smaller. So you won't have nice video quality, but at least you will keep receiving telemetry.

Ideally you want telemetry and SSH packets to be prioritized over video packets. Unfortunately there doesn't seem to be an easy way to achieve this. It might be possible with packet marking and routers that support QoS, but I haven't gone all the way down that rabbit hole.

Instead, I came up with a relatively simple but effective method to limit negative effects of temporarily reduced bandwidth. When packets are queuing up in the 4G modem, it adds latency. This is very visible when sending ping commands from the RPi to the computer. When the data limit is approached, the latency slowly builds up, indicating that the queue is filling up. My script basically pings the computer every second. If the latency is too high, the bitrate is reduced. When the latency is sufficiently low, the bitrate is increased again.

The script changes the bitrate automatically by updating the bitrate value in mediamtx.yml. MediaMTX has a hot-reload feature, which will detect those changes and apply them immediately.

While I forgot to make a copy of the original script, it is pretty straight-forward to implement with this description. I think I actually had ChatGPT write it for me originally.

Ardupilot dangerous failsafe behavior

After reading the docs many times and going through the source code, I came to the shocking conclusion that it is impossible to configure failsafe properly for this scenario. Maybe this has been addressed in a more recent version, but as of 4.3.5 this was not possible.

What I wanted to do, and what seems completely logical to me, is the following: The failsafe (return to home) should only trigger when both the RC link and the telemetry link are lost. But in Ardupilot we can only choose to trigger failsafe on either RC loss or telemetry loss. In order to fly out of RC range, I had to configure this to activate failsafe on telemetry loss. But that is very dangerous for some scenarios:

If I fly the plane in FBW-A mode for landing, while having perfect control with the RC link, a loss in 4G connection can trigger a RTH-failsafe! This is not an unlikely scenario at all, because the 4G connection could drop when descending below the tree line.

The work-around is slightly better, but still really bad. To prevent this scenario, I deliberately trigger the failsafe by disconnecting the ground station before landing. That puts the airplane in RTH mode. From there I can take control with the RC link and land the plane, and I can be assured that no failsafe will be triggered. However if I would lose RC link in this scenario, the airplane would be doomed...

If I remember correctly, PX4 implements the failsafe in the way you would expect. Therefore at this point I can really not recommend Ardupilot for 4G flights, unless you are willing to modify the failsafe behavior yourself. I'm sure they'd be open for a pull request.

Mapping 4G coverage and signal quality

To get a better understanding of the signal quality, I created a script on the RPi to save all signal parameters together with coordinates and altitude to a csv file. I forgot to make a copy of the final script, so you'll have to recreate it with this description. Or ask ChatGPT.

Even though I dislike python very much, I couldn't get around the availability of some libraries to put together a simple script. I used the huawei-lte-api for easy access to signal data, without having to worry about login cookies etc. I used pymavlink to subscribe to position data on GLOBAL_POSITION_INT. I would keep a copy of the latest position information, then poll the huawei api every second to get the signal data. This signal data together with latest position is written to a csv file.

This csv can then be analyzed for some interesting insights. It allowed me to compare the performance of different 4G modems for the same flight trajectory. I also identified a new 4G tower that was not visible in open-source datasets. And also it gave me some insights into the switching between towers, which often seemed to be the cause for a brief drop in connection.

Invite others to watch along with WebGCS

The cool thing about an internet-connected airplane is that you can literally fly it from anywhere in the world. And it would be even cooler if you don't even need QGroundControl for that.

Out of frustration with QGroundControl (keeps crashing when plugging in a joystick) and Mission Planner (I am allergic to Windows), I decided to build a simple web interface myself. Initially this is just a visualization tool. I could do some cool custom stuff with the map, like drawing extra layers to mark the fences and security checkpoints. QGroundControl is lacking a feature to show course-over-ground on the map, so I added that too.

While this is currently just a viewer, there is a lot of potential to turn it into a full-blown ground control station. I'm continuing to work on this under the name of WebGCS. It will be an open-source project, making it easy to customize to your specific use-case. If you have any specific applications in mind that you need help with, you can get in touch with me through my company Mapture.ai.

Continue Reading