Push Waze Traffic Info through RDS-TMC?

[rhodesman]

EDIT: It appears Waze does not share it's traffic data via their API. Now I'm looking into Google's traffic data which does have an available api.

NOTE: In an effort to organize some cic hackery, I'm starting this new thread for figuring out a way to push waze's traffic data into the cic via the RDS-TMC system that's already active. So please keep posts in this section dedicated to that effort.

I do like the traffic system on the iDrive and it works okay. Only issue I have is that Waze gets down to the side street level where the RDS-TMC system only seems to cover major highways or roads. This works fine for road trips but when trying to navigate city traffic to find the quickest route home, the CIC falls short. This sucks because the CIC has so much more detail when it comes to vehicle position, direction and doesn't get "lost" when GPS is blocked by a large building, overpass, or tunnel. It would be awesome if I can use the GPS routing of the CIC with the data of the waze traffic system!

Here are some options I think could lead to a usable hack:

• build a raspberry Pi that broadcasts the waze traffic data over the RDS-TMC FM band
• tap into the Sirus/XM connection coming in through the receiver to overwrite the traffic info received from space
• find how the cic is storing and accessing the traffic data and edit/overwrite that data on the cic directly through the USB port using shell scripts.

Let me know if any of this sparks some ideas and further ways to hack this to our advantage.

 

[emoRaivis]

Great idea! I was looking for a way to inject waze data into the navigation and didn't even think of RDS-TMC, since it's not used in my country at all.

Here's a excerpt from the "BMW CIC System design specification":

TMC data are provided by the AMFM Tuner using the MOST Co Standard FBlock 0x41 (TMC Tuner), version 2.3.2, contents. The tuner delivers raw data, filtered by the navigation instructions set by MOST Command “TMCSelectionFilter (FktID 0x203)”
​

So it should be possible to simply inject the TMC data through MOST. I think that @dolphin78 mentioned in another thread, that he found a way to communicate with MOST.

 

[rhodesman]

So... I have this MOST to USB connector for my ICom system, if there is a way to get drivers for my PC/Android to directly connect to it then we would have a direct connection to the MOST for TMC data connection. Need to do some more research!

Pic for reference:

 

[rhodesman]

@Xer0449 @emoRaivis @BLACKHAT @doublespaces

I have progress...sort of and am at the point where I need someone else's help who understands the things I don't.

to recap:

• I have setup a Raspberry Pi to grab it's current location from a GPS dongle
• feed those lat and long coordinates to a script that pulls the waze traffic and alert data for that location
• I have a couple scripts running to save that JSON output into a more organized XML file
• and this is where I get stuck!

I am using GNURadio with the GR_RDS plugin to possibly stream the data via FM to my car. This might be too much for the Rasp-pi to handle and am looking at a couple alternatives (headless options being key!) but for now I'm just trying to get it work, I'll worry about performance later. However my problem lies in I'm not sure how (or what format) I need to convert the JSON/XML traffic data into proper TMC broadcast data. The nice thing about the GR_RDS plugin is that it will handle the encryption part itself so that's cool, but it's getting the data ready for GR_RDS that I'm stuck. I've probably read every white paper on TMC that is publicly available on the web but nothing I am finding is helping me understand how to parse the data for streaming.

I need help!!

because we all like sneak peeks, here is a snippet of JSON output from Waze when I set the coordinates to LA (because at my house, traffic is not as constant )...

{ country: 'US',
  city: 'Los Angeles, CA',
  severity: 4,
  delay: 2000,
  speed: 4.933333333333334,
  type: 'Small',
  line:
   [ { x: -118.288079, y: 34.037016 },
     { x: -118.294444, y: 34.03709 },
     { x: -118.303872, y: 34.0372 },
     { x: -118.30927, y: 34.037262 },
     { x: -118.311463, y: 34.037264 },
     { x: -118.313336, y: 34.037201 },
     { x: -118.31507, y: 34.037084 },
     { x: -118.3163, y: 34.036981 },
     { x: -118.317684, y: 34.036804 },
     { x: -118.322783, y: 34.036068 },
     { x: -118.325804, y: 34.035646 },
     { x: -118.327242, y: 34.035491 },
     { x: -118.328861, y: 34.035401 },
     { x: -118.338281, y: 34.034939 },
     { x: -118.343473, y: 34.034692 },
     { x: -118.357134, y: 34.034048 },
     { x: -118.359869, y: 34.03392 },
     { x: -118.360468, y: 34.033906 },
     { x: -118.361279, y: 34.033916 },
     { x: -118.362169, y: 34.033965 },
     { x: -118.362691, y: 34.034012 },
     { x: -118.363714, y: 34.03415 },
     { x: -118.366961, y: 34.034804 },
     { x: -118.369403, y: 34.03533 },
     { x: -118.375239, y: 34.036555 },
     { x: -118.37695, y: 34.036934 },
     { x: -118.377362, y: 34.037004 },
     { x: -118.377775, y: 34.037039 },
     { x: -118.378336, y: 34.037071 },
     { x: -118.378833, y: 34.037058 },
     { x: -118.379824, y: 34.036957 },
     { x: -118.380288, y: 34.036863 },
     { x: -118.380932, y: 34.036684 },
     { x: -118.381383, y: 34.036513 },
     { x: -118.382221, y: 34.036115 },
     { x: -118.386057, y: 34.033852 },
     { x: -118.391987, y: 34.030359 },
     { x: -118.393436, y: 34.029717 },
     { x: -118.394154, y: 34.029498 },
     { x: -118.394759, y: 34.029365 },
     { x: -118.395377, y: 34.029257 },
     { x: -118.396022, y: 34.029207 },
     { x: -118.396684, y: 34.029197 },
     { x: -118.396684, y: 34.029198 },
     { x: -118.397236, y: 34.029219 },
     { x: -118.399602, y: 34.029432 },
     { x: -118.409201, y: 34.030299 },
     { x: -118.409725, y: 34.030401 },
     { x: -118.410445, y: 34.030585 },
     { x: -118.411085, y: 34.030794 },
     { x: -118.412328, y: 34.031244 } ],
  street: 'I-10 W',
  roadType: 3 }
{ country: 'US',
  city: 'Los Angeles, CA',
  severity: 3,
  delay: 378,
  speed: 8.891666666666667,
  type: 'NONE',
  line:
   [ { x: -118.338437, y: 34.034739 },
     { x: -118.329629, y: 34.035181 },
     { x: -118.327329, y: 34.03531 },
     { x: -118.325863, y: 34.035441 },
     { x: -118.322423, y: 34.035924 },
     { x: -118.318362, y: 34.036507 },
     { x: -118.31714, y: 34.036649 },
     { x: -118.314249, y: 34.036933 },
     { x: -118.313284, y: 34.036984 },
     { x: -118.311428, y: 34.037047 },
     { x: -118.309114, y: 34.03704 },
     { x: -118.303868, y: 34.036986 },
     { x: -118.294257, y: 34.036868 },
     { x: -118.286724, y: 34.036788 } ],
  street: 'I-10 E',
  roadType: 3 }
{ country: 'US',
  city: 'Los Angeles, CA',
  severity: 3,
  delay: -1,
  speed: 0,
  type: 'NONE',
  line:
   [ { x: -118.317639, y: 34.02506 },
     { x: -118.317635, y: 34.023634 } ],
  street: 'Arlington Ave',
  roadType: 7 }
{ country: 'US',
  city: 'Los Angeles, CA',
  severity: 4,
  delay: 496,
  speed: 6.655555555555556,
  type: 'NONE',
  line:
   [ { x: -118.303276, y: 34.08664 },
     { x: -118.298092, y: 34.083042 },
     { x: -118.296615, y: 34.081944 },
     { x: -118.295758, y: 34.081389 },
     { x: -118.295569, y: 34.081293 },
     { x: -118.294504, y: 34.080799 },
     { x: -118.293305, y: 34.080248 },
     { x: -118.291774, y: 34.079353 },
     { x: -118.29123, y: 34.079059 },
     { x: -118.290615, y: 34.078787 },
     { x: -118.290143, y: 34.078607 },
     { x: -118.289488, y: 34.078412 },
     { x: -118.28903, y: 34.078317 },
     { x: -118.288496, y: 34.078258 },
     { x: -118.286786, y: 34.078113 },
     { x: -118.28607, y: 34.078024 },
     { x: -118.285399, y: 34.077929 },
     { x: -118.282406, y: 34.077278 },
     { x: -118.280331, y: 34.076763 },
     { x: -118.279392, y: 34.076507 },
     { x: -118.279149, y: 34.076458 },
     { x: -118.278723, y: 34.076347 },
     { x: -118.278318, y: 34.076236 },
     { x: -118.277695, y: 34.076047 },
     { x: -118.275269, y: 34.075369 },
     { x: -118.273772, y: 34.074915 },
     { x: -118.273059, y: 34.074651 },
     { x: -118.272258, y: 34.074316 },
     { x: -118.271324, y: 34.073874 },
     { x: -118.270301, y: 34.073364 },
     { x: -118.269269, y: 34.072926 },
     { x: -118.264377, y: 34.070814 },
     { x: -118.263699, y: 34.070526 },
     { x: -118.263382, y: 34.070396 },
     { x: -118.262565, y: 34.070093 },
     { x: -118.261348, y: 34.069664 } ],
  street: 'US-101 S',
  roadType: 3 }

Also, looking at the JSON output from Waze, I cannot figure out the representation the "speed" number has. Is it a multiplier, a fraction, a percentage? I have no idea.

 

[rhodesman]

small update:

I did find this in a PDF from the TMC forum dated Oct 1, 2005. Only thing I need now is how the JSON/XML should be structured for broadcast and GPS understanding.

 

[Xer0449]

It looks like it's compiled...?

https://www.blackhat.com/presentati...resentation/bh-usa-07-barisani_and_bianco.pdf
https://en.wikipedia.org/wiki/Radio...eband_coding_structure_from_the_RDS_bitstream

I wonder if the TMC antennae/receiver is a device under /dev/ that you could cat out. Would be useful for testing.

 

[rhodesman]

It looks like it's compiled...?

https://www.blackhat.com/presentati...resentation/bh-usa-07-barisani_and_bianco.pdf
https://en.wikipedia.org/wiki/Radio...eband_coding_structure_from_the_RDS_bitstream

I wonder if the TMC antennae/receiver is a device under /dev/ that you could cat out. Would be useful for testing.

I can cat out the FM antenna but I don't get anything when doing that.

My current progress is that I have the Waze traffic data parsed into an OSM JSON or XML format. My main problem is now how to get that sent over the Fm broadcast via the Raspberry Pi. I was trying to use PiFmRds but I can't seem to figure out how to get it to accept and broadcast the TMC data. Maybe my formatting of the TMC data is wrong?

Here is a snippet of the OSM Json data that should be ready to send via TMC:

{
    "type": "FeatureCollection"
    , "features": [{
        "type": "Feature"
        , "geometry": {
            "type": "LineString"
            , "coordinates": [[-118.330838, 33.987757], [-118.330834, 33.988132], [-118.330834, 33.988239], [-118.330836, 33.988585], [-118.330838, 33.988826], [-118.330835, 33.988995], [-118.330821, 33.990865], [-118.330809, 33.993122], [-118.330799, 33.994968], [-118.330798, 33.995089], [-118.330788, 33.997741]]
        }
        , "properties": {
            "street": "Crenshaw Blvd"
        }
    }, {
        "type": "Feature"
        , "geometry": {
            "type": "LineString"
            , "coordinates": [[-118.317131, 33.989102], [-118.317126, 33.989885], [-118.317202, 33.990017]]
        }
        , "properties": {
            "street": "S Van Ness Ave"
        }
    }
}

 

[rhodesman]

one more thing, I found this file on the CIC hdd. not sure if it's the actual TMC stored data or not, but it's in the /tmp/ folder and the most recent update to that file was today. So I suspect it's possible. only problem is I have no idea how to read it!! Nothing want to open it :/

 

[doublespaces]

I bought one of these(There is F30 version too) so android should be even easier to access for me, including waze:

https://avinusa.com/8-8-bmw-3-series-e90-e91-e92-e93-android-multimedia-navigation-system.html

 

[rhodesman]

Okay, I'll hand it to you, that is really cool! But, hacking the CIC and bending it to my will is still my #1 goal.

 

[doublespaces]

Okay, I'll hand it to you, that is really cool! But, hacking the CIC and bending it to my will is still my #1 goal.

Yeah, only a few people are going to spend $550 on something like this, I'm mostly doing it for the touch screen. The specs of the android portion are a bit disappointing, 1gb ram, 1.6ghz and android 4. I would have paid more for a modern, octacore design with 2gb ram in a heartbeat, though I really shouldn't have to for $550. But I'm going to run with it, best touch screen option even though its the only option.

I got the final pieces(I hope) for my combox retrofit, so that should be finished up shortly. Even though, the benefits of the combox look pretty bleak considering this latest modification.

 

[rhodesman]

Not what I originally intended but this is getting me closer to my end goal. At the very least I am getting an active GPS location, pulling data from waze for said location and parsing it to JSON. I then have a slew of scripts I wrote to filter certain events and display them how I want. Traffic congestion and flow info is being parsed into XML for TMC broadcast, road notifications (like potholes or accidents) are being parsed into RDS-TMC XML notifications, and Police notifications are being displayed via the 16x2 LCD shown here as a makeshift "radar detector" (but I have not been successful at getting the RDS-TMC data to broadcast properly). Basically, as of now I have a "non-radar" radar detector! LOL.

 

[Xer0449]

Woah, that's awesome! Great work!

What was your hold up? Was the antennae not picking up the broadcast?

 

[toxx]

I activated RDS-TMC in my CIC but no joy. Don't know for sure if in UAE/Dubai there is some free TI service over the commercial radio waves. It was easy to do it, just VO CODE the CIC with $6UH . I will try this when my combox arrives, given that it'll have internet from tethering maybe i can setup RTTI over that to see something on the screen.

I digged the net for finding SDK/sources for Android Auto, maybe it can be ported to CIC to use it ? For sure Apple Carplay is not available for study.. But i couldn't find nothing for AA also.

On my wife's Chevy Trax 2017, i want to admit that Android Auto is better than Car Play. At least you can run Waze on it and it works really smooth. More applications, also.

 

[toxx]

Not what I originally intended but this is getting me closer to my end goal. At the very least I am getting an active GPS location, pulling data from waze for said location and parsing it to JSON. I then have a slew of scripts I wrote to filter certain events and display them how I want. Traffic congestion and flow info is being parsed into XML for TMC broadcast, road notifications (like potholes or accidents) are being parsed into RDS-TMC XML notifications, and Police notifications are being displayed via the 16x2 LCD shown here as a makeshift "radar detector" (but I have not been successful at getting the RDS-TMC data to broadcast properly). Basically, as of now I have a "non-radar" radar detector! LOL.

You might want to find a module fo RPi that broacasts in FM range and also has RDS encoder. As far as i know, old style TMC uses FM RDS signal carrier to transport the data to the car. (which is a 38khz carrier near the main wave, first 19khz is stereo if i remember correctly) Then you can set it up in your garage to inject tmc traffic coming from your scripts through RDS datafeed.

 

[rhodesman]

You might want to find a module fo RPi that broacasts in FM range and also has RDS encoder. As far as i know, old style TMC uses FM RDS signal carrier to transport the data to the car. (which is a 38khz carrier near the main wave, first 19khz is stereo if i remember correctly) Then you can set it up in your garage to inject tmc traffic coming from your scripts through RDS datafeed.

Now you're thinking with portals...err I mean radio waves! ;P

You mean an RTLSDR like this one???

Fun fact: those bars to the left and right of the audio broadcast of 101.9 in my area are in fact, the RDS-TMC data stream!!!

Yes, the end goal is to have the RaspPI broadcast the traffic info via FM...In fact, I can technically do that now. The only real hiccup I have is that TMC is a closed and "paid" system here in the US. It's broadcasted encrypted (but that's already been hacked) however the way the data is setup for the stream is unknown to me. I just need to find the right format to parse the traffic data into and then I'm done! That's my only problem, not knowing how to organize the traffic data into a format that the iDrive is expecting so it can use it!

If anyone has any info about that, you know, I'm all ears!!!

 

[Deadknight]

Now you're thinking with portals...err I mean radio waves! ;P

You mean an RTLSDR like this one???
View attachment 6010
Fun fact: those bars to the left and right of the audio broadcast of 101.9 in my area are in fact, the RDS-TMC data stream!!!

Yes, the end goal is to have the RaspPI broadcast the traffic info via FM...In fact, I can technically do that now. The only real hiccup I have is that TMC is a closed and "paid" system here in the US. It's broadcasted encrypted (but that's already been hacked) however the way the data is setup for the stream is unknown to me. I just need to find the right format to parse the traffic data into and then I'm done! That's my only problem, not knowing how to organize the traffic data into a format that the iDrive is expecting so it can use it!

If anyone has any info about that, you know, I'm all ears!!!

Is it possible to push radio data through radio device (/dev) if it exists?

For me finding how bmw gets rtti data is more interesting.

 

[rhodesman]

Is it possible to push radio data through radio device (/dev) if it exists?

For me finding how bmw gets rtti data is more interesting.

I'm with you there, I was just going after the low laying fruit first!

 

[rhodesman]

Is it possible to push radio data through radio device (/dev) if it exists?

For me finding how bmw gets rtti data is more interesting.

I'll tell you what, if you can tell me how the RTTI data is setup & organized (xml,json,csv,osm,etc.) I can parse the waze traffic data into that. Then it's only a matter of having the CIC take our waze traffic data vs. BMW's traffic data.

Currently my code for getting and parsing the waze data is running directly on a local raspberry pi but I can easily move it to a public server where it's accepting incoming GPS coordinates and spitting back the waze traffic data for that location. In fact, if I port it to an android and/or iOS app, then you would be able to just use your phone as the GPS and pushing the data back either through your phone or through the highjacked CIC data connection if we ever can do that!

Hmmm... in fact, I may just move to this setup as I'm already pushing the raspberry Pi's processing power to it's limits trying to have it do multiple processes and calculations at one time. Currently it takes about 20-30 seconds from getting the GPS signal to retrieve the waze data, parse the data, run a bunch or queries on that data and parse it into various forms based on what it finds and spits it back. If I dropped this code on my 12 core 4GHz server with 128GB of RAM, that would drop that time down to a second or two, if that!!

 

[Deadknight]

I'll tell you what, if you can tell me how the RTTI data is setup & organized (xml,json,csv,osm,etc.) I can parse the waze traffic data into that. Then it's only a matter of having the CIC take our waze traffic data vs. BMW's traffic data.

Currently my code for getting and parsing the waze data is running directly on a local raspberry pi but I can easily move it to a public server where it's accepting incoming GPS coordinates and spitting back the waze traffic data for that location. In fact, if I port it to an android and/or iOS app, then you would be able to just use your phone as the GPS and pushing the data back either through your phone or through the highjacked CIC data connection if we ever can do that!

Hmmm... in fact, I may just move to this setup as I'm already pushing the raspberry Pi's processing power to it's limits trying to have it do multiple processes and calculations at one time. Currently it takes about 20-30 seconds from getting the GPS signal to retrieve the waze data, parse the data, run a bunch or queries on that data and parse it into various forms based on what it finds and spits it back. If I dropped this code on my 12 core 4GHz server with 128GB of RAM, that would drop that time down to a second or two, if that!!

I don't have rtti subscription, i cannot collect data to trace :\. I'm mobile developer, real problem in mobile is making phone talk to cic.

When i have time i will look at the binaries that you sent me. They might have connection strings in it.