2017-11-03, 16:40 (This post was last modified: 2017-11-03, 16:42 by cutty.)
(2017-11-03, 15:57)Alanpenwith Wrote: Hi All
I realise that storms are moving phenomena but the movement is relatively slow and my thought was that there might be a simple way of tuning a detector. Well tuned detectors must benefit the network as a whole. Not everyone would have the time to analyse a lot of historical data and, like me, many have probably not learnt enough to use it effectively.
Maybe something along the lines I suggest would be a sort of 'Quick Start Guide' for those with limited time and/or knowledge run in parallel with the 'proper tools' if the server capacity would allow it. Brian has pointed me to the Strikes versus Signals graphic which gives a guide as to what I am doing but not what I should be achieving in comparison to stations which should be showing similar results.
As recently we have has storm clusters distant from each other I have been able to improve my performance somewhat by watching other stations fairly close to me. What I am suggesting is a formalisation of that.
I see the point of Egon's grading system but the spread of the system must mean there are some who 'don't have the time or knowledge/lost interest' who be be stuck in group one but might be motivated by a simple method of improving.
Regards
Alan
Now, think a moment... We don't 'tune' these systems or antennas.. we do optimize
Decide whether your want a high effectivity / efficiency number. That would indicate 'station' performance.
My suggestion does NOT relate to 'network' figures... only what an operator has decided.
With today's network density, except for those fringe areas, where stations might elect to go 'group TWO' if they can stay 'efficient'...
why would 'high signals' low stroke' stations not desire to be more accurate. efficient, effective, within a 'smaller radius'?
My systems normally are operating with 'radius' of 30-1200 KM --- and the only thing that knocks my numbers down, as a station,
is my dadgum sporadic noise makers that don't trigger interference mode. In my location with my environment, for me to
send 60000 signals trying to hit 'South America' as I once could, years ago, is foolish, and does NOTHING for network except
send a bunch of signals that won't be used for locating anyway...
So as far as my 'station / location' 'quality' my station can NEVER perform as it used to, or is capable of if some local parameters
were changed... So, I simply optimize down for better overall performance, since the network does NOT work any better with
all my extra signals, even if they're strokes, greater than about 1200km.
I don't know if this has been mentioned here yet, but a station's efficiency, ratios, etc. can vary from very high to very low depending on where the storms are in relation to it. Perhaps that should be taken into account?
(2017-11-03, 15:57)Alanpenwith Wrote: Hi All
I realise that storms are moving phenomena but the movement is relatively slow and my thought was that there might be a simple way of tuning a detector. Well tuned detectors must benefit the network as a whole. Not everyone would have the time to analyse a lot of historical data and, like me, many have probably not learnt enough to use it effectively.
Maybe something along the lines I suggest would be a sort of 'Quick Start Guide' for those with limited time and/or knowledge run in parallel with the 'proper tools' if the server capacity would allow it. Brian has pointed me to the Strikes versus Signals graphic which gives a guide as to what I am doing but not what I should be achieving in comparison to stations which should be showing similar results.
As recently we have has storm clusters distant from each other I have been able to improve my performance somewhat by watching other stations fairly close to me. What I am suggesting is a formalisation of that.
I see the point of Egon's grading system but the spread of the system must mean there are some who 'don't have the time or knowledge/lost interest' who be be stuck in group one but might be motivated by a simple method of improving.
Regards
Alan
Yes. There is a thread "Should we have a FAQ?" but it hasn't had any activity for weeks.
(2017-11-03, 16:52)mwaters Wrote: I don't know if this has been mentioned here yet, but a station's efficiency, ratios, etc. can vary from very high to very low depending on where the storms are in relation to it. Perhaps that should be taken into account?
Doesn't that proposed suggestion accomplish that very thing? It DOES NOT look at 'network'... only the station's activity.
Hi
Sorry if I used the wrong term. By tuning I meant setting the station up to the best advantage of the network.
So IF it were possible to pick a storm and see the results of stations likely to agree with my attempt to optimise to 1200km it gives me something to work on. As I see it I don't need or want to compare my attempts with stations seeing 5000km strikes.
It is more or less what I have been doing and it seems to be working. I have watched the live maps and my signals chart to see if stations around me are reacting similarly. It works after a fashion, it's fairly quick and easy - my stats have improved so my idea was only to see if the principal had any mileage.
Sometimes a person seeing things with a different perspective can spark a new train of thinking - or maybe not !
Regards
Alan
@Cutty: first problem with the proposed figures is that they do not discriminate between real sferics and noise. The signal sent category contains both and we would first like to reduce noise
second problem is what mwaters wrote: the position of storms both affects the signals sent and the strokes detected, but in different ways. For my station at the moment very often the storms are in a favorable position and I get sometimes very high participation in the detected strokes, so the stations performance looks good. When the storms are in a less favorable position relative to my station, the performance looks lousy. So you could judge the station either as "excellent" or as "lousy", depending where the storms are.
Third problem is that many, many stations send signals of real sferics that are much distorted by noise. That their signal can be used anyway looks like a miracle to me.
Another serious problem has been spelled out clearly by Alanpenwith: we need tools that show us quickly how a station is performing without looking at lots and lots of actual signals from our and other stations. I do not think that this can be achieved with only these two numbers proposed.
I propose to make a list of examples for good sferic signals and also for interference noise, and for clean signals and noisy signals, together with hints how to improve the situation. People could then browse this catalog and check how their station is performing.
In addition waterfall plots would help, histograms of signal strength of sent and used signal, etc.
2017-11-03, 20:24 (This post was last modified: 2017-11-03, 20:30 by readbueno.)
Pasense, In the small storms that we have had in the North Atlantic for the last 48 hours or so I have been actively monitoring eight stations in the area of interest.
The "signal to stroke ratio" for these stations varies from on average, only over this time period, from 2 signals with 1 stroke, to 8200 signals for three strokes:
i.e. In the last hour and taking the figures hourly your station's figures are 9,725 signals sent 76 stroke recorded. This give the basic ratio of about 130:1 for your station
My figures over the same period are 135 signals sent for 63 strokes recorded this ratio is 2.14:1.
I have only included stations that were recording strokes in this storm, Their squares were lighting up on the map.
They were all Blue stations running ferrite antennas and E field of reasonable size (100 mm).
There is a big disparity between the figures that somehow needs to be explained.
These are stations in reasonable geographical area, using the same equipment, so how come the big difference in ratios, between signal sent and strokes recorded, i.e. 2:1 to 2600:1?
This was based on average, over a short amount of time, using a small sample, but these are not the worst nor the best stations on the graphs or in the lists either.
If this sort of measurement was taken for all stations, which the server could probably do, much easier than one person just doing the sums on a calculator, we would have a better picture about the state of the network in any given period and geographical area, in relation to the amount of lightning activity and ambient noise.
With this information we could then see where our individual station rates are over a different period of time and the affect of any adjustments that we make.
Taking short "snapshots" and saying, "Yes, today I am near the top or near the bottom of the list." is very volatile and changes from minute to minute and is maybe not the best way of assessing either the stations performance, regardless of what it is called, nor probably is it a good way to establish the "relative health" of the network.
I have asked similar questions to you about What is an acceptable signal, and about various forms of noise and interference, in these forums, and I have read the forums in Wx Blitzortung as well, I have responded to the "Should we have a FAQ" forum etc., etc.
Yes, it would be nice if someone with the skills could make new tools or continue in the development of the tools that we do have.
This also takes time and we have to be clear what we want the tools to do.
Basically, we must each take responsibility for our own stations, within the guidelines established, or to be established, and make the best possible adjustments and positioning of our stations as we can.
Not everyone is in an ideal situation in regards to this and we should not expect perfection, however, even given the average stations limitations and the "time" commitments of the station owners, we should be able to do better than what the graphs and the quality of some of the signals in the archives are showing.
I have made several suggestions, both in these forums and in conversation with several individual stations.
They amount basically to this:-
1. Reduce your gains to a much lower level, yes, you may lose a few distant strokes, but your overall efficiency will be much higher.
2. Increase the thresholds to a good level, this will stop many forms of annoying interference triggering the station, and "local lightning is always going to exceed the threshold anyway.
3. Make use of the station itself as a tool to find your local sources of noise.
This can be greatly aided with a Long Wave AM radio, tuned to a reasonably unpopulated frequency and just walking around your house and your locality noting what the prominent sources of noise and interference are.
At least some of these will be in your control and you should be able to deal with these.
Some may be in the public domain, Noisy streetlights and power lines, etc, these should be reported to the appropriate body.
Some other may not be able to be fixed, although a polite letter or patient conversation may mitigate some of these sources caused by your neighbours.
Some noise and interference is not reducible at source Traffic noise, factories, Military installations etc, and you will have to resort to filters on the control board itself. or using different gains and threshold at times of high noise.
Even in these situations you can still record a lot of lightning, using the nulls on your antenna or shielding, but you will have to live within limitations that others may not have.
If you try just these three things you may be pleasantly surprised!
We are still pioneering and like all pioneers to a certain extent we are making the map as we go along, but at least a good path has already been established and although it is not plug'nplay (Where's the fun in that? You might just as well watch the public screens!) it is not too much of a challenge either.
We still have a long way to go, and a lot of things to do, and I am sure that we will all grow as individual and the network will be even better than it is.
Enjoy, do your best and learn from each other, we all have different skills, and sharing in this activity is a good thing by itself.
Highest regards,
2017-11-03, 21:07 (This post was last modified: 2017-11-03, 21:08 by pasense.)
Readbueno wrote .............................................................................................................. I have made several suggestions, both in these forums and in conversation with several individual stations. They amount basically to this:- 1. Reduce your gains to a much lower level, yes, you may lose a few distant strokes, but your overall efficiency will be much higher. 2. Increase the thresholds to a good level, this will stop many forms of annoying interference triggering the station, and "local lightning is always going to exceed the threshold anyway. 3. Make use of the station itself as a tool to find your local sources of noise. ------------------------------------------------------------------------------------------------------------- to 1 taking a look at http://wwlln.net/n showing the thunderstorms active on earth, you may notice that there is a lot of action within reach of the Canary Islands, especially towards the west and south. Attenuation over sea is low, so the sferics can be recorded well here. You should note the the station is also receiving sferics from the Americas and contributing to the location there. In South America and Africa there are simply not enough stations yet so many real signals the station here receives are not yet used by the network. 2.) I have no indication whatsoever of local noise sources. There simple is no "annoying interference" triggering the station, the signals are almost completely from sferics.
3.)There are no local sources of noise of relevance. The only one I have not really managed to trace concerns only the E field, hence the low amplification on this channel.
The bottom line is that I chose to run an "island station" aiming at coverage of the Atlantic around and from the figure below you can see that it works: up to a distance of 1000 km the efficiency is 100%, gradually dropping to 80% at 4000km. Then the American continent comes into play and the recognition rate drops. You chose to run a station optimized for local recognition and that's what makes the difference. A quality measure of an island station is the S/N ration, where N means real noise and not sferics from distant strokes, and of course the location accuracy. The efficiency and efficacy measure are of little value here. Cutty has already proposed to make this distinction between isolated stations and integrated stations.
Pasense, That is exactly what I am saying, your station is not one of the less productive stations and seems to be within the better half of the graph.
The stations that my advice is mainly aimed at are the stations at the bottom right hand corner of the graph some of whom are sending tens or even hundreds of thousands of signals per hour and only recording very few strokes.
I am also part of the Region three network and report many strokes there as well, the graph that you chose to show is at the height of the lightning season here.
maybe you would compare that with today's graph, after all we don't wish to be accused of comparing Apples and Oranges.
You appear to have no local activity recorded at all?
Maybe there is not any lightning to ever be recorded in the sunny Canary Islands unlike in rainy Galicia?
readbueno,
since I started the station here there was no thunderstorm on the Atlantic nearer than about 1000 km, and some scattered thunderstorms over North Africa, so there was simply no local lightning at all. This may change when we get lows from the west, with lightning and the sorely missed rains, but at the moment nothing like this is in the forecasts.
(2017-11-03, 21:48)pasense Wrote: readbueno,
since I started the station here there was no thunderstorm on the Atlantic nearer than about 1000 km, and some scattered thunderstorms over North Africa, so there was simply no local lightning at all. This may change when we get lows from the west, with lightning and the sorely missed rains, but at the moment nothing like this is in the forecasts.
I can see this discussion going round and round in circles and not getting any where if we aren't careful. I think it's time for a summary and way forward:
1) There isn't a set of guides for new participants. Guides on the subjects of:
Overview - What a station should aim to do, taking into account its location, to provide the network with high quality data.
Controller - Construction, description of how it works, description of each firmware setting, option, mode, etc.
Power - Noisy SMPSU's, cures, PoE, Linear supplies.
Antennas - Construction, mounting, connection, location, use of their chracteristics.
Enviroment - Common interference/noise sources, tools and methods to trace noise sources.
Settings - Ways to interpret the station information available from blitzorgtung, LMO, MyBlitzortung. The effects and side
effects of hardware pararmeters.
All stuff for the FAQ, much of it is in the forum(s), if it can be found.
2) The current station measures (Effectivity S, M and L) are not particulary useful aids to setting up or optimising a station. Only one alternative(*) has been suggested that has had some disscussion. Namely Cutty's:
Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected
Probably time to apply to data sets from a range of real stations to check that it does return useful and meaningful values.
(*) If there have been others they have been lost in the noise by this reader.
@allsorts,
thanks for the fine summary; only one thing I would like to add: better tools for analysis like waterfall and signal strength histogram.
If it is the consensus that we need a FAQ with this content, I will wait until Sunday evening UTC and then contact Egon directly.
2017-11-04, 08:53 (This post was last modified: 2017-11-04, 09:35 by readbueno.)
Hi Dave, et al, There was also Alan's suggestion of Geographical clusters, Cutty's "Three group system" and my suggestion that time and some form of averaging/weighting should be included in the calculation. (RMS/Time? x strokes + signals sent /100) or something?
Yes, I think we have enough basic and not so basic questions to actually start the FAQ and maybe that would be the way to handle it, a very basic public and possible open FAQ, "Ten things you want to know about Lightning Detection!" type, and a more detailed and restricted FAQ for stations with a matching forum for discussion of new or debatable questions, your 7 or 8 categories seem to cover most things and a perhaps a catch-all miscellaneous section as well.
When does it open for business?
2017-11-05, 14:43 (This post was last modified: 2017-11-05, 14:47 by allsorts.)
For the "station measure" the KISS Principle needs to be applied (Keep It Simple Stupid). The current blitzortung station list Effectivity isn't very fair, take these numbers for my and the next closest three stations pulled from the station list earlier:
Cutty's
Sigs Parti Ef L Effc Efct
4306 1814 16% 0.42 0.73
1229 619 1% 0.50 1.01
1016 330 0% 0.32 0.48
494 341 0% 0.69 2.23
As these numbers come from the station list they cover the previous 60 mins and up to 5000 km. The vast majority of those strikes are happening over Itally, 1000 miles (1600 km) away. There does need to be a time period over which the signals and particpated strikes are counted. An hour seems about right for a quick "how is my station doing" and if updated every minute ought to reflect any settings changes fairly rapidly and totaly after an hour, which isn't too long to wait. I think there should also be a longer period, 24 or 48 hours perhaps, that would give an indication of how the station is doing over the longer term. Storms would generally come and go over that time scale.
If we do adopt Cutty's suggestion there does need to be at least a divide by zero trap and/or some cap on the value of "Effectivity". A cap of 10 seems reasonable, to get a Effectivity of 10, 91% of the signals sent would have to be used. Also this value is exponential but I think this helps us as the sensitivity of the value increases with the percentage of signals used. That is 50% of signals used gives a value of 1, 60% 1.5, 70% 2.33, 80% 4, 90% 9. A target of getting over 66% (ie a value greater than 2) of signals used ought to be achievable by most stations. The 2.23 and 2.54 above represent 69% and 72% used.
Distance, geographic clusters, etc aren't really relevant to a stations "performance measure". We want that measure to be a useful aid in adjusting a stations settings (mainly gain and threshold) to achieve the goal of maximum participation with minimum signals sent. The station operator decides on the range of their station following the recomended guidelines of isolated - longer range - higher gain, lots of neighbours - shorter range - lower gain. The threshold is then adjusted to a level where local interference only rarely triggers the sending of a signal. This may mean that an isolated station with a noisy local enviroment doesn't have the range that the guidelines suggest it ought to have to give maximum benefit the network. That is solved by reducing the local noise, if possible, if not "it is, what it is".
2017-11-05, 17:07 (This post was last modified: 2017-11-05, 17:11 by readbueno.)
Hi Dave, I think that does not sound too bad. The actual cap/weighting and the number of signals used data, should be calculated using data that is available from the server over various periods of time, and IMO the longer periods do not need to be "real-time," i.e. they could be calculated and stored only once over that time period.
I still think that geographical clusters might serve some useful purpose, but maybe they aren't so important.
For several days the results in the station lists do not appear to be updating for several stations and in many cases near strokes are not registered at all? This could be a software bug.
Although, I seem to have very low "ping" rates and over the last few days Traceroute has shown a lot of hops within the internet network, with very long delays, so this might be part of the problem.
At the height of the storm over southern Spain and Italy, the archives did not seem to produce any graphics, even for strokes that were several hours old, was this data lost?
These things would also matter if you were adjusting the station, just using the raw data.
The, more or less, ten percent of stations on the far right and, especially the ones at the bottom right, of the meteomelin graph (the stations sending 10,000+ signals per hour, for very few strokes.) should have a closer look at how their station is functioning and make the appropriate adjustments and mitigate their sources of interference as much as is possible.
Maybe these stations should be excluded from the server during busy periods automatically.
It is not all noise that triggers interference mode and maybe the criteria for this needs to be reassessed.
These stations could be jamming up the server, when others are actually trying to send real data.
My 2 cents.
(2017-11-05, 17:07)readbueno Wrote: Hi Dave, I think that does not sound too bad. The actual cap/weighting and the number of signals used data, should be calculated using data that is available from the server over various periods of time, and IMO the longer periods do not need to be "real-time," i.e. they could be calculated and stored only once over that time period.
Well what ever is adopted will be handled by the server as the current system is. I agree though that for evaluation the data ought to come from a carefully crafted URL request to the server. I did check that, for my station, the "Participated" count in the station list agreed with what MyBlitzortung said, which they more or less did.
I think the longer periods should still be a rolling period, ie the last 24 hours, updated every 5 minutes. If you had 24 hours updated every 24 hours the first data could be nearly 48 hours old: Update at T0, first data is T-24 hours old. Just before the next update at, T+24, the first data is 24+24 = 48 hours old.
(2017-11-05, 17:07)readbueno Wrote: I still think that geographical clusters might serve some useful purpose, but maybe they aren't so important.
Geography probably has a place to check how your station is doing in a general way. If you assume that sferics are going to be fairly similar within, say, 50 km of the station. Large differences in Signals and Participated would indicate a problem of some sort. Relatively high level of local noise, forcing the threshold up, a poorly setup station, causes being on either yours or the other station(s).
(2017-11-05, 17:07)readbueno Wrote: Maybe these stations should be excluded from the server during busy periods automatically.
(Stations sending 10's of thousands of signals/hour but only participating for very few of those signals).
Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.
2017-11-05, 19:04 (This post was last modified: 2017-11-06, 10:50 by readbueno.)
(2017-11-05, 18:57)allsorts Wrote: Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.
There are the Alerts, if these are set up and people checked them, they could be used?
There is also the considerable "waste" of data.
On a suggestion, this evening I tried my station on full Auto, in that three hours, my station sent more signals than in the previous 96 hours and the data rate shot up from less than 15 Mbytes per day to over 1Gbyte per day.
All this extra data for a very marginal return in a few extra strokes recorded?
I have also discovered that UDP does not have a complete communication "handshake," like TCP does and that if data suffers packet loss the station just continues sending data regardless of any acknowledgement, and the server is not even aware that large chunks of data may in fact be missing. With TCP each block of data has to be received correctly before the next set of packets is sent, In UDP this is not the case and there is no redundancy.
While this important fact improves gameplaying by not repeating data requests, thus speeding up the game and filling in from previous data, the lost data just being exactly that LOST!
In a lightning detection and recording system, however, the network could in fact be losing vast amounts of data, due to overload or other reasons and this is not so acceptable?
There is probably, as in most systems, an optimal acceptable rate for data sent and data lost, however by overstressing the system by sending tens or even in some cases hundreds of thousands of extra signals per hour, when this is mainly useless noise, is not an efficient nor effective way to run the network.
Internet Protocols are designed for different types of usage and if we are to have complete archives and mapping then this would also need to be addressed.
(2017-11-05, 18:57)allsorts Wrote: Excluding the data from these stations doesn't help with them "clogging up" the link(s) into the server. Unless they are told to shut up the packets will still be sent and the server do something with them even if it's just drop them based on IP address. If we can come up with a good "performance measure" that could be used via the regular "phone home" system to tell a station to shut up. The hard part is how do you inform the station operator that their station has been told to shut up? email I guess, if a valid email is known. A "message waiting" flag in the controller web interface might not be seen for weeks.
There are the Alerts, if these are set up and people checked them, they could be used?
There is also the considerable "waste" of data.
On a suggestion, this evening I tried my station on full Auto, in that three hours, my station sent more signals than in the previous 96 hours and the data rate shot up from less than 15 Mbytes per day to over 1Gbyte per day.
All this extra data for a very marginal return in a few extra strokes recorded?
I have also discovered that UDP does not have a complete communication "handshake," like TCP does and that if data suffers packet loss the station just continues sending data regardless of any acknowledgement, and the server is not even aware that large chunks of data may in fact be missing. With TCP each block of data has to be received correctly before the next set of packets is sent, In UDP this is not the case and there is no redundancy.
While this important fact improves gameplaying by not repeating data requests, thus speeding up the game and filling in from previous data, the lost data just being exactly that LOST!
In a lightning detection and recording system, however, the network could in fact be losing vast amounts of data, due to overload or other reasons and this is not so acceptable?
There is probably, as in most systems, an optimal acceptable rate for data sent and data lost, however by overstressing the system by sending tens or even in some cases hundreds of thousands of extra signals per hour, when this is mainly useless noise, is not an efficient nor effective way to run the network.
Internet Protocols are designed for different types of usage and if we are to have complete archives and mapping then this would also need to be addressed.
Brian.
@Brian, TCP demands network discipline all the way and NAT specific policies.
By the way, lot of setting to the Blue system can be done remotely by the server: all amp levels, thresholds, fine tune of filters, etc. However this will require rethinking all security policy, introduction of TLS, etc. And servers must become little smarter, some of the AI required to adapt the network of sensors per given sferics conditions / regions. I'm sure @Egon is playing with some code already
Alerts: Not looked into them very much, if they send an email to the operator that would work. Relying on the operator looking at the right part of the web interafce or the LEDs isn't good enough. I sometimes don't loook at the web interface for weeks and the controller is up in a loft out of eye and ear sight...
Yes, UDP is "send and forget", however I think all the data for a single signal fits into a single packet of 1.5 k bytes ish. So you don't need the overhead of the TCP error checked/retransmission system to ensure that all the data of a signal arrives. It all does, or doesn't if the packet gets lost. Assuming any packet loss is random acros stations, losing packets isn't going to have a great impact on the network. Say 50 stations report a stroke and 15 are needed to located it, 35 can get lost...
