+ Reply to Thread
Results 1 to 3 of 3

Thread: Searching for Metrics at the End of the Continent...

  1. #1
    Points: 55, Level: 1
    Level completed: 10%, Points required for next Level: 45

    Location
    Seattle, WA
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question Searching for Metrics at the End of the Continent...




    Hello everyone,

    I'm a systems engineer working for the railroads building tools to manage a nation wide network of 1.5m band packet radios. I also do a whole lot of audio recording and engineering work when I'm not busy doing that. And I have four kids. And a lot of guitars. And a banjo.

    I love data (lots of it) and the radio network is providing me lots of opportunities to analyze performance, but I've come across a problem and I'm pretty well stuck: I'm trying to predict the weather.

    No, seriously! An inversion layer can have serious effect on radio wave propagation by creating a "tropospheric duct." If you're a HAM operator, you might think these are cool as they can let you receive and transmit from very far away. If you're running a multiplexed packet network, it can be problematic for the exact same reason. You end up with radios in Texas stomping on a radio in Minnesota's time slot.

    That's where I'm at right now. I get updates from every radio in the field every 15 minutes that includes all the other radios it can hear and how far away they are. I am currently using that data to construct coverage areas (a polygon in a GIS) for the radio and plotting them on a map. Since I store the polygon as an object, it gives me the ability to analyze those coverage areas for other purposes.

    There is no deterministic way to declare a ducting event. So my thinking was that I could simply analyze the polygons, and if I saw them getting larger in a short amount of time, I could use that as a trigger for further analysis and provide a weighted value about our guess as to the probability of a tropo event taking place.

    The problem is "getting larger" is a really, really relative thing. Coverage in the mid-west where it is flat can be really big, while coverage in the Sierra-Nevada's can be quite small. Adding a single radio to a mid-west radio's coverage might increase its extent by a hundred miles, while the same addition of a single radio in the mountains might gain you another mile or so of coverage.

    I have three bits of data I can work with.

    1. The number of radios a particular radio can hear.
    2. The maximum extent of it's coverage area. i.e. the farthest radio from it that it can hear.
    3. The area of the coverage polygon (a convex hull around the positions of the radios in #1.)

    I'm working with the last three hours of reports for any given radio. My first try at this was a simple linear regression of the maximum extent for each report. If my line had a positive slope, it was growing. If not, I could ignore it. The problem with this was with radios that temporarily added a new radio, and then dropped it again. This leads to a positive slope even though the outliers are no longer there. It works for some cases, but not all. I tried polynomial regressions, and basically ended up with the same problem. The dispersion of data in these sets is fairly broad so I get good sized standard deviations and regressions with not so big correlation coefficients.

    Then I started looking at the coefficient of variation from report to report. If I see my standard deviation starting to grow (because we added a new radio way far away) then the CV would shrink. But if we added a radio way far away, then the mean would also grow, which would cause CV to grow. I haven't completely abandoned this line of thinking yet, but I'm not sure I understand what CV is telling me. I do know it is dimensionless, which is what I need. I just don't know if CV is it.

    So now I'm here looking for some expert help. Is there a better method for doing this correctly (of course there is!)? It seems like such a simple question to ask of data "Did that coverage area get much bigger than normal over the last three hours?"

    If you have an answer, or some guidance, you'll have my eternal gratitude, and you may just help keep the trains running on time.

    Many thanks!

    -Jerome

  2. #2
    Beep
    Points: 82,851, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Ames, IA
    Posts
    12,749
    Thanks
    300
    Thanked 2,585 Times in 2,207 Posts

    Re: Searching for Metrics at the End of the Continent...

    Could you provide some sample data. Maybe some from each of the scenarios you describe so we have a better idea of what you're working with?
    I don't have emotions and sometimes that makes me very sad.

  3. #3
    Points: 55, Level: 1
    Level completed: 10%, Points required for next Level: 45

    Location
    Seattle, WA
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Searching for Metrics at the End of the Continent...


    Sure.

    Here's a sample of the table I'm working with. This is for radio ID 4718921. The "f_radio_id" is the "Far radio" that it can hear. It provides me with a range and bearing so I know where it is located.

    I project that position and then create a convex hull around the set of points. That's my coverage area. You can see from one load to the next that the same radios "generally" show up and that the're around 40~60 miles out, but occasionally it will pick up a radio as far out as 160 miles. That's the one that screws me up.

    I have retain 7 days worth of this data, and I retain the polygons indefinitely.


    update_time | radio_id | f_radio_id | dbm | distance | bearing | id | channel
    ---------------------+----------+------------+------+----------+---------+---------+---------
    2017-01-05 00:00:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3917416 | 114
    2017-01-05 00:00:09 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3917416 | 114
    2017-01-05 00:15:09 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3918269 | 114
    2017-01-05 00:15:09 | 4718921 | 4718923 | -90 | 60.4 | 271 | 3918269 | 114
    2017-01-05 00:30:10 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3919119 | 114
    2017-01-05 00:30:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3919119 | 114
    2017-01-05 00:45:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3919964 | 114
    2017-01-05 00:45:10 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3919964 | 114
    2017-01-05 00:45:10 | 4718921 | 4718930 | -109 | 95.97 | 129 | 3919964 | 125
    2017-01-05 01:00:10 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3920811 | 114
    2017-01-05 01:00:10 | 4718921 | 4718923 | -88 | 60.4 | 271 | 3920811 | 114
    2017-01-05 01:15:09 | 4718921 | 4718923 | -88 | 60.4 | 271 | 3921661 | 114
    2017-01-05 01:15:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3921661 | 114
    2017-01-05 01:15:09 | 4718921 | 4718930 | -109 | 95.97 | 129 | 3921661 | 125
    2017-01-05 01:30:11 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3922513 | 114
    2017-01-05 01:45:09 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3923361 | 114
    2017-01-05 01:45:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3923361 | 114
    2017-01-05 02:00:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3924212 | 114
    2017-01-05 02:00:10 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3924212 | 114
    2017-01-05 02:15:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3925058 | 114
    2017-01-05 02:15:10 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3925058 | 114
    2017-01-05 02:30:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3925906 | 114
    2017-01-05 02:30:10 | 4718921 | 4718924 | -111 | 164.17 | 281 | 3925906 | 125
    2017-01-05 02:30:10 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3925906 | 114
    2017-01-05 02:45:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3926756 | 114
    2017-01-05 02:45:09 | 4718921 | 4718923 | -90 | 60.4 | 271 | 3926756 | 114
    2017-01-05 02:45:09 | 4718921 | 4718930 | -109 | 95.97 | 129 | 3926756 | 125
    2017-01-05 03:00:08 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3927616 | 114
    2017-01-05 03:00:08 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3927616 | 114
    2017-01-05 03:15:09 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3928464 | 114
    2017-01-05 03:15:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3928464 | 114
    2017-01-05 03:15:09 | 4718921 | 4718930 | -109 | 95.97 | 129 | 3928464 | 125
    2017-01-05 03:30:10 | 4718921 | 4719196 | -72 | 45.34 | 127 | 3929314 | 114
    2017-01-05 03:30:10 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3929314 | 114
    2017-01-05 03:45:09 | 4718921 | 4718930 | -108 | 95.97 | 129 | 3930163 | 125
    2017-01-05 03:45:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3930163 | 114
    2017-01-05 03:45:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3930163 | 114
    2017-01-05 04:00:09 | 4718921 | 4718930 | -109 | 95.97 | 129 | 3931012 | 125
    2017-01-05 04:00:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3931012 | 114
    2017-01-05 04:00:09 | 4718921 | 4718924 | -110 | 164.17 | 281 | 3931012 | 125
    2017-01-05 04:00:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3931012 | 114
    2017-01-05 04:15:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3931856 | 114
    2017-01-05 04:15:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3931856 | 114
    2017-01-05 04:15:09 | 4718921 | 4718930 | -108 | 95.97 | 129 | 3931856 | 125
    2017-01-05 04:30:09 | 4718921 | 4718923 | -89 | 60.4 | 271 | 3932704 | 114
    2017-01-05 04:30:09 | 4718921 | 4718930 | -108 | 95.97 | 129 | 3932704 | 125
    2017-01-05 04:30:09 | 4718921 | 4719196 | -71 | 45.34 | 127 | 3932704 | 114

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats