Results of a 2 day road trip with the Pi black box’s GPS data

GPX file of Vermont trip with car black box

On a recent Vermont trip, I had the black box Pi setup and running. It was the standard setup as shown here on the site with GPS logging. In 2 days of driving from MA to upstate VT I logged 550 miles in the car and the Pi logged it all.

Vermont maple syrup sign

Ignore the 2 straight lines as those are the result of a bug in the PHP script used to generate the GPX files. You can use it here with your own GPS log.

There were over 30,000 GPS data points recorded over the trip. The default is 10 times a second to pull GPS data from the receiver and store it to the local SQLite3 database on the Pi before it’s synced to the server and removed from the device. In the future this would allow not only different frequency (which is user configurable upon setup of the Python code) but also an “automatic” mode that increases frequency at slower speed and decreases at higher speeds.

 

 

 

GPX and KML output

So far, my results have been mixed when outputting GPX files from my GPS data. But the GPX export feature on this site is working to spec. Next up will be to use some of the incredibly detailed information from Google to generate KML files too. Then I can output the test data I’m getting from my Pi onto Google Maps.

As with the GPX exporter, the KML exporter will use PHP on the server to read from the GPS data in MySQL to generate the XML file.

Motorcycle dash cam and GPS logger

2014-06-14 16.20.39-2

Motorcycle riding is a major passion of mine. I ride as much as I can during the spring/summer/fall months. And since I’m a major proponent of ATTGAT, that means I’m fully geared up for every possible weather scenario. On a recent drive out to Keene, NH, I came across some beautiful scenery and saw some wonderful things. My favorite was a family of baby ducks and their mother trying to cross the road. The mother kept battling to get past my line because cars kept driving past without stopping. Finally I was the only vehicle to stop and let them cross. First off, what major jerk doesn’t stop to let a family of baby ducks cross a street? Second, I realized how great it would’ve been if I had a dashcam to record the incident.

That’s when I remembered that the Black Box Pi is just as applicable to a motorcycle as it is to a car. In fact, since I’m big on trip logging, the GPS logging would be even more fun than the more day-to-day utility of doing it for a car.

Keep in mind one of the big advantage to logging GPS data through a motorcycle black box like this instead of through your smartphone is that your phone’s battery isn’t being drained at such a drastic rate. My battery already barely lasts me a full day on the bike. Going in and out of service works the phone more as it tries to find a signal. Add GPS logging, and you’ve got yourself a dead phone pretty quick.

Tonight I’m going to go out and buy another Pi, another USB GPS receiver and a cheap webcam. Once I get access to the 12 volt accessory port on the bike, it’s a trivial process to hook it all up and get it going. Stay tuned for details and instructions as soon as I get this done. Here’s everything I’m buying tonight to get this started.

Motorcycle dash cam and GPS logger

Everything listed below is available from Amazon.com except the last item (Pi UPS) which is produced by a German company. There’s a US distributor that carries it.

  • Raspberry Pi (Model B)
  • 32 GB SD Card
  • USB Micro cable
  • AC/USB adapter
  • GPS receiver
  • Pi UPS

 All the items above work out to around $140 total.

 

Raspberry Pi Power Supply

The Raspberry Pi requires 5v of power through a 5v micro USB cable. Like any other computer, you can’t just pull the plug on power without potentially damaging the storage as well as the hardware. In this case, repeated unplugging of the cable can corrupt the SD card and over time, damage the components as well. In our application of using the Pi in a car, it will be powered on and off according to the ignition switch. This presents a major problem for anyone using the Pi with their car. When you shut off your car, power to the Pi via your car’s 12v cigarette lighter will cease. There are some exceptions and some cars’ ports continue to provide power. Even if that’s your case, you still will want the system to shut down once your car’s not running.

Raspberry Pi Battery Backup

The solution to using a Pi in a vehicle application is the Pi UPS. It’s a $30 component out of Germany that acts like any other UPS (Uninterruptible Power Supply) does for your computer.

It’s similar in functionality to what you might already have in your home or office:

apc-back-ups-es-750

What the Pi UPS does is allow for the Pi to shut down safely each time you turn off your car.  When you hit the ignition switch, power is suddenly lost to the Pi and the Pi UPS senses that. It switches seamlessly to battery backup with the Pi noticing. It then sends a signal to the Pi to shutdown. The whole process takes 20 seconds by default and can be shortened.

Mine only took about 5 minutes to setup. It’s really easy to do and requires no tools at all. Their guide is pretty straightforward. After setup, you plug the USB cable into the Pi UPS instead of the Pi. That’s all there is to it.

Raspberry Pi with backup power supply and UPS

2014-06-08 13.39.26

Raspberry Pi power supply attached to the Pi UPS

2014-06-08 13.39.51

First test of GPX file output

Today was the first trial run with the Raspberry Pi “car-puter” running in my car during my commute. I’ve run it a few times before with little success due to the sudden power loss from shutting the car down. This is the first time I had the Pi running using the Pi USP from CW2. It allows the Pi to shut down properly using backup power after the main power (from the car) shuts down when the car is parked.

The GPS data was run through a PHP script to generate the GPX XML file and uploaded to http://www.gpsvisualizer.com/map_input to visualize it.

GPS_Visualizer__Google_Maps_output

The results are on the right track. The horizontal line is likely drawn by the mapper because of a gap in data. As for the rest of the track, it looks about right. It’s just shifted over by a few degrees of Lat/Lon. Going to have to review the GPS calibration.

A few Facebook friends were confused as to why I was logging my GPS data from somewhere on the south shore where I don’t even live, all the way onto a boat in Mass Bay and back. heh.

A key thing I learned is how to continue working on the GPS time issue. Should the GPS locations be logged according to the Pi’s local clock? Probably not as that clock doesn’t work since the Pi doesn’t have an internal battery. The GPS receiver pulls UTC which is more universally accurate than worrying about the local computer’s clocks being correct.

GPX file output high level architecture

A big part of this project isn’t just about collecting car data such as OBD2, GPS, and more; but rather, it’s about getting it back while. While my focus is solely on data collection, I want to be able to export the GPS data into a standard file format to use in mapping programs. The GPX file format is a simple XML spec that’s easy to generate.

GPX Output Process

Version 1.0

Build the GPX export server-side. Take as inputs: User ID (API Key or email address), date range for the GPS data.

Version 2.0

Build the GPX export process directly on the Pi itself, or on a smartphone app or other client. This’ll require adding a JSON web service to the server to spit back relevant GPS data upon request.

OBD2 Cloud Storage Architecture

In January 2014 after working on the recent Pi based OBD2 logger, I’ve finally published the storage architecture. The OBD2 Cloud Storage project is on GitHub here:

https://github.com/BizDevGeek/OBD2CloudStorage

This project contains the API and the MySQL database. The API is in PHP and uses JSON + REST. It connects to a local MySQL database to store all the OBD2 PID data which a client device can log via the internet and the API calls. A Python SDK is also available on GitHub. I plan on adding SDKs in PHP and then Java (for Android) and ObjectiveC (for iPhones). The SDK is initially in Python because of pybod which is an open source OBD2 reader in Python.

Playing with a Pi for the car

In late 2013 I began seriously putting together a handful of Raspberry Pi projects. Here’s a detailed writeup of the most promising of them: using a Pi as a car black box. The Pi can handle multiple sensors, store that data, and get it uploaded to a central cloud storage service.

The database and Big Data lover in me wants data, lots of it. So I’ve gone with building a black box for my car that runs all the time the car is on, and logs as much data as I can capture. This includes:

  • OBD2
  • GPS
  • Dashcam
  • and more

Once you’ve got a daemon running, and the inputs are being saved then the rest is all just inputs. Doesn’t matter what it is. It’s just input data.

My initial goal is to build a blackbox that constantly logs OBD2 data and stores it to a database. Looking around at what’s out there for OBD2 software, I don’t see anything that’s built for long term logging. All the software out there is meant for 2 use cases: 1)live monitoring 2)tuning the ECU to get more power out of the car. What I want is a 3rd use case: long term logging of all available OBD2 data to a database for analysis.

In order to store all this data I decided to build an OBD2 storage architecture that’s comprised of

  • MySQL database
  • JSON + REST web services API
  • SDK that existing OBD2 software would use to store the data it’s capturing
  • Wrapping up existing open source OBD2 capture data so it runs as a daemon on the Pi
  • Logging data to a local storage buffer, which then gets synced to the aforementioned cloud storage service when there’s an internet connection.

Right now I’m just doing this for myself. But I’m also reaching out to developers of OBD2 software to gauge interest in adding this storage service to their work. In addition to the storage, an API can be added for reading back the data such as pulling DTS (error) codes, getting trends and summary data, and more.

The first SDK I wrote was in Python. It’s available on GitHub. It includes API calls to register an email address to get an API key. After that, there are some simple logging functions to save a single PID (OBD2 data point such as RPM or engine temp). Since this has to run without an internet connection I’ve implemented a buffer. The SDK writes to a buffer in local storage and when there’s any internet connection a background sync daemon pulls data off the buffer, sends it to the API and removes the item from the buffer. Since this is all JSON data and very simple key:value data I’ve gone with a NoSQL approach and used MongoDB for the buffer.

The API is built in PHP and runs on a standard Linux VPS in apache. At this point the entire stack has been built. The code’s nowhere near production-ready and is missing some features, but it works enough to demo. I’ve built a test utility that simulates a client car logging 10 times/second. Each time it’s logging 10 different PIDs. This all builds up in the local buffer and the sync script then clears it out and uploads it to the API. With this estimate, the client generates 100 data points per second. For a car being driven an average of 101 minutes per day, that’s 606,000 data points per day.

The volume of data will add up fast. For starters, the main database table I’m using stores all the PIDs as strings and stores each one as a separate record. In the future, I’ll evaluate pivoting this data so that each PID has it’s own field (and appropriate data type) in a table. We’ll see which method proves more efficient and easier to query. The OBD2 spec lists all the possible PIDs. Car manufacturers aren’t required to use them all, and they can add in their own proprietary ones too. Hence my ambivalence for now about creating a logging table that contains a field for each PID. If most of the fields are empty, that’s a lot of wasted storage.

Systems integration is much more of a factor in this project than coding each underlying piece. Each underlying piece, from what I’ve found, has already been coded somewhere by some enthusiast. The open source Python code already exists for reading OBD2 data. That solves a major coding headache and makes it easier to plug my SDK into it.

There are some useful smartphone apps that can connect to a Bluetooth OBD2 reader to pull the data. Even if they were to use my SDK, it’s still not an ideal solution for logging. In order to log this data, you need a dedicated device that’s always on when the car’s on and always logging. Using a smartphone can get you most of the way there, but there’ll be gaps. That’s why I’m focusing on using my Pi as a blackbox for this purpose.

http://www.jozefnagy.com/content/playing_pi