+- Blitzortung.org Forum (https://forum.blitzortung.org)
+-- Forum: Public Forums (https://forum.blitzortung.org/forumdisplay.php?fid=29)
+--- Forum: General Discussion (https://forum.blitzortung.org/forumdisplay.php?fid=31)
+--- Thread: Efficiency (/showthread.php?tid=1261)
RE: Efficiency - cutty - 2015-07-23
(2015-07-20, 10:04)Gerhard.Wittevee Wrote:(2015-07-20, 09:21)Knickohr Wrote: The best way to tune a station is to optimize it in this way that every detected signal is a stroke !
But not every stroke is registered by a single station. Try to optimize "quotas" Locating ratio and Stroke ratio at lightningmaps.org. A good starting is, if both values has nearly the same value, whether they are low or high.
Your station doesn't have the claim receiving all strokes (in your area) !
Thomas
Thanks Thomas,
I am aware of the fact that there are more ways of optimizing my stations. But this does not answer the questions that I posted here.
Gerhard
Gerhard, perhaps one way, long term, to optimize, is follow the stats on LMO, Advanced, etc... someday I'll figure it all out myself... but I like to look at 'deviation'... it might seem to me that I'm providing optimized data the better the deviations, at least for the 'good signals' I send...
RE: Efficiency - Gerhard.Wittevee - 2015-07-23
(2015-07-23, 18:49)Cutty Wrote: Gerhard, perhaps one way, long term, to optimize, is follow the stats on LMO, Advanced, etc... someday I'll figure it all out myself... but I like to look at 'deviation'... it might seem to me that I'm providing optimized data the better the deviations, at least for the 'good signals' I send...
Thanks Mike,
I am checking the LMO graphs too. But my lack of maths-skills is annoying
The main reason for using the participants-table for optimizing the station is the relatively quick reaction you see to a change in settings. In an hour the "new" results are visible, and can be compared to other stations. So a increase or decrease in lightning activity is also visible, the surrounding station wil increase or decrease by roughly the same amount. But with the current calculation those values are harder to compare. Up to a point where stations with higher signals and lower strikes are rated higher. That can not be right in my humble opinion.
Again, with my limited maths-skills i am certainly not able to design a better calculation. I totally understand why it is a good thing to use the signals in te calculation, but the old calculation gave me more grip on checking and optimizing my settings. But I sure am glad this topic is now the talk of the town
I hope a great idea pops up!
Gerhard
RE: Efficiency - Knickohr - 2015-07-24
OK guys,
update the graphs. Sorry for confusing
Thomas
RE: Efficiency - micha.d - 2017-10-23
ok, I'm relatively new and I read this thread to find out what I should actually optimize for. I'm setting up a station that shall be moved to India later. So my focus is to get all the sferics that are far away because there are no stations near by. And by the way I agree to a lot of arguments what the stations should be optimized for.
Now regarding the calculations: By default the station - once located there - might be getting good values because only detected strikes of that station will be "known" and so only those will be taken into consideration for calculation. Also there will be no other stations to compare with, so I'm asking myself what I should look for as soon as the station is there.
On the other hand I tried to compare the station with other stations around here while preparing it and it's pretty useless because they should optimize for different criteria, as meantioned above.
So what should I do? Well I faced one problem and I think that LMO (for my station https://www.lightningmaps.org/blitzortung/europe/index.php?bo_page=statistics&bo_show=station&lang=en&bo_station_id=14922) nearly has the solution. There I have a signal view that is close to the internal one, but it also includes frequency analysis to find potential noise. Now still it's hard to distinguish between good and bad signals in detail. Will a filter setting that prevents noise lead to signals that cannot be used for detection? What is the best amplification? Do I have noise and from where?
And this is me feeling: For me the best help would be to have some information coming along with this display. So for every signal I can see there I would appreciate to get this data:
* was it used to detect a stroke?
* if yes, how far away was the stroke? Which direction?
* how many other stations were involved?
Again this is my feeling. Any ideas on that? Is it possible?
And one really off topic point. It would be helpful to optimize the mechanism to adjust the gains automatically. If the environment changes (e.g. TV or light on), then the signals get more noise. As soon as they reach the threshold I'm currently changing some settings because I think the automatic mode does not work for me very well. But then I have completely different efficiency which confuses me very much and does not help anymore at all. The approach I proposed above should still help to observe the signals. Then with this new display everyone should be able to optimize for his location in different situaltions. In the end - even when I can't influence the automatic - I can find some settings for different situations and decide offline which one to prefer and to keep as a static setting.
In India this noise challange will be much more I expect, because the invironment is a crowded city and conditions will change a lot. So the proposed analysis is one part, tuning the automatic could be an even more efficient part.
RE: Efficiency - orion_jb2001 - 2017-10-23
Interesting fresh observations. The automatic feature of this TOA system has never, I don't think, been fully utilized.
RE: Efficiency - allsorts - 2017-10-24
(2017-10-23, 20:33)micha.d Wrote: There I have a signal view that is close to the internal one, but it also includes frequency analysis to find potential noise. Now still it's hard to distinguish between good and bad signals in detail. Will a filter setting that prevents noise lead to signals that cannot be used for detection?
It's certainly possible but you'd have to be really vicious with the filters. Due to a couple of VLF transmitters within 30 odd miles I have the lo-pass option fitted and set to 17 kHz. Still have plenty of signals used by the system.
(2017-10-23, 20:33)micha.d Wrote: What is the best amplification? Do I have noise and from where?
Amplification, what ever works for your station and its location both local and regional. When the station is out in India you'll be wanting to cover as large and area as possible but being in a city may find that local conditions prevent the use of higher gains.
Noise, you'll know it when you see it. Strikes are random, similar but all different impluses withe the "sombreo" shape. Noise/interference may be constant or have relatively long on periods centered on given frequency(s) (SMPSU's in wall warts, CFL or LED bulbs). A regular pulse (electric fence). A single spike in the frequency plot or a visible sine wave in the signal plot (VLF transmitter).
(2017-10-23, 20:33)micha.d Wrote: And this is me feeling: For me the best help would be to have some information coming along with this display. So for every signal I can see there I would appreciate to get this data:
* was it used to detect a stroke?
* if yes, how far away was the stroke? Which direction?
* how many other stations were involved?
Again this is my feeling. Any ideas on that? Is it possible?
If you are logged into LMO and viewing your station the Archives > List gives distance direction and number of participants. What I have not see it show is if a given signal was used. There is a "Participated" field but I've never seen it.
You could try downloading and using the MyBlitzortung.php script. I've hacked that on the Archives > Search result page to give me a count of how many of the displayed strikes my stations signal was used for.
Automatic is quite slow in settling down on a set of gains etc, thinks days rather than hours, though it does track local enviromental changes quite rapidly.
RE: Efficiency - readbueno - 2017-10-24
Hi Micha, Using full auto is not a good idea, it is better to set the station for its normal environment and then use "Auto adapt to noise" and the "Amplitude filters."
Even in India the lightning is going to be the biggest signal within a reasonable area and the number of station is growing, slowly but surely.
Low to medium gains and a reasonable threshold without going too much into "interference mode" except for very close lightning should probably still be the aim, rather than trying for distance.
Even here in Europe, there are far to many stations that seem to think that setting the gains high and the thresholds low and running in "Auto" will be O.K.
Looking at the better managed stations, They seldom have gains above 8*8 and many are even lower. The threshold should seldom be below 100.
This is a good tool for helping to find a good balance, between signals sent and strokes recorded.
https://www.meteomelin.be/staticfiles/blitzortung_tools/signals_vs_strikes/?region=1&id=
The best stations tend towards the top and the left and the "worst" stations towards the right hand side and the bottom right corner.
This does depend on the conditions at the time and there will be some movement, however some stations appear to be consistently doing well within the middle area.
I hope that you find this helpful. I look forward to seeing a new station always, but the first one in India will be special.
Brian.
With Europe and North America now moving into the quieter time of the year, now is a good time to try and get the best possible figures for signals to strokes,
If there is no Lightning near you then your station should not be sending thousand of signals per hour
...
RE: Efficiency - allsorts - 2017-10-24
Full Auto works well here, if I use manual seetings I have to keep an eye on it and tweak the settings very day or so. Is there much difference between "Automatic" mode and "Manual" with Auto Adapt to Noise and Amplitude filter set to on?
I like that plot, my station is currently nicely in the upper middle area. B-) And consdiering the nearest activity is 1500 miles away SE of Italy out over the Med not bad, it's not really trying hard either gains 10*10 threshold around 85 mV and noise around 35 mV. It's participated in 211/1000 strikes over the last 1h51m within 2000 miles. Drop the range to 1000 miles and it's 9/18 over 10h30m...
RE: Efficiency - Egon - 2017-10-26
Hi Folks,
I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
/Egon
RE: Efficiency - mwaters - 2017-10-26
(2017-10-26, 21:03)Egon Wrote: Hi Folks,
I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
/Egon
This is wonderful news, Egon!
May I suggest you include the signals vs strikes ratio? The closer to 1:1, the better.
I use this to see how my station compares with others who consistently do well.
RE: Efficiency - micha.d - 2017-10-26
Thanks for the answers. I'll only respond to the things related to efficiency.
(2017-10-23, 21:30)orion_jb2001 Wrote: Interesting fresh observations. The automatic feature of this TOA system has never, I don't think, been fully utilized.
. Well that's an argument for optimizing the auto as well, not only the efficiency display.Isn't it possible to distinguish repeated and sporadic signals. All the repeated are noise. If really many station go for auto, then that might be a quick win.
(2017-10-24, 01:04)allsorts Wrote: It's certainly possible but you'd have to be really vicious with the filters. Due to a couple of VLF transmitters within 30 odd miles I have the lo-pass option fitted and set to 17 kHz. Still have plenty of signals used by the system.
That's a pretty good info. That gives me a good feeling that the noise can be managed. It's hard to find that out through the efficiency display. At least I did not make it so far.
(2017-10-24, 01:04)allsorts Wrote: If you are logged into LMO and viewing your station the Archives > List gives distance direction and number of participants. What I have not see it show is if a given signal was used. There is a "Participated" field but I've never seen it.
You could try downloading and using the MyBlitzortung.php script. I've hacked that on the Archives > Search result page to give me a count of how many of the displayed strikes my stations signal was used for.
Well, I don't see the mentioned data in one source, maybe I missed something. Either it's all stations (not just mine) or I don't get the "used" flag for my station.
I thought about writing a tool that downloads all the stats but all that requires effort and is only usable locally in the end. I would want to pimp the display for all members.
(2017-10-24, 06:00)readbueno Wrote: This is a good tool for helping to find a good balance, between signals sent and strokes recorded.
https://www.meteomelin.be/staticfiles/blitzortung_tools/signals_vs_strikes/?region=1&id=
I like this one, but it's also a "slow" efficiency marker like all the others. But again we could take some lessons learned out of it. Close to the diagonal is good, far away is bad. I think that is a part of the current calculation already.
(2017-10-26, 21:03)Egon Wrote: I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
Ok, this is maybe only a starting point, but what would have helped me in the beginning is an advice what to change. Let's do some examples:
- The first point mentions to check for repeated signals. If they occur the advice could be to lower the gain or increase the threshold. I see this as a clear rule for manual or automated adjustments. It protects the server and is more relevant in regions with a lot of station. So the number of stations within 1000km could be taken into account. Detection of repeated signals can be done on server or client side to distribute the load. As soon as repeated signals are gone, this indicator is blind.
- After following the first advice I might have to look for the source of the noise to get better. I cannot imagine an auto for removal of the noise but maybe a noise comparison helps to understand why my station is not going to be on top of the list. The advice would be: reduce noise, increase distance to the noise etc.
- If the station has noise of a specific frequency, then the filters could be adjusted automatically. This might change over the day, so it's also a case for the automatic. All the frequency analysis graphs of the last n minutes could be accumulated to clearly see the noise frequency in the graph. That could show us if it's really a specific frequency that is there. Then the noise could be eliminated automatically by adjusting the frequency of the filters or asking the user to do it. We could even think about eliminating a specific frequency by notch filter calculations on the server. This is expensive because it must be done before using the signal.
- The second example above is the distance to the diagonal of the graph. There is no clear advice for me what to do. If I look at the graph, then several station are where they should be, the few others more down right should be having the noise problem above, right? They have a lot of signal without strikes. That is noise and noise is often repeated. So the indicator whould tell where you are and how important your noise problem is. This also works if you don't have much noise.
- The last question for the stations on the diagonal whould be: Should / Can they be more at the top right? You will only get to the top right with low noise. But you will also only get to the top right if there are strikes nearby. Did I get this right? If yes, I think a factor for each strike needs to be calculated. Nearby strikes count more, far away less. Don't forget the intensity is decresed with the square of the distance, right? I don't have a clear formula yet, but in principle, if strikes are nearby (value high), then it's easy and I should be top right. If strikes are faaaaar away (value low), then it's ok to be more left at the bottom. Calculating the ratio of the strikes value and the real detects should prevent that stations are going up and down all the time in the ranking. The limit is always the noise (again). To get a formula, we would have to keep some stations with stable settings and environment (including the noise) and play with parameters of our formula until they have a stable value over time I guess.
.
RE: Efficiency - readbueno - 2017-10-26
Hi Egon, Stations should maybe be weighted as to how useful the signals are that they send to the server?
If there are only 20 stations that locate a stoke is that data more valuable than when 350 stations receive the same stroke?
Is a stroke recorded at 200 km as useful as one at 1200 km or 4500 km?
At present the effectivity of a station is heavily biased towards distance, Effectivity L. Would it be more efficient to base the stations position on their shorter and medium effectivity?
Could there be some explainable scientific basis for these measures?
Many stations appear to have a very poor signals sent to strokes recorded ratio.
I have noticed this as over 75,000:1 in some cases, these stations still appear higher in the lists than stations with more even ratios.
Should Detecting stations whose signals are actually used in the calculations for the location of the stroke be worth more than a station that detects the stroke, but is not one on which the calculation is based.
Should stations in less densely populated parts, Blitzortung wise, get a premium for detecting strokes in relation to the area that they are trying to cover, i.e Coastal stations or stations that by their position cover mainly underpopulated areas? The Pacific Ocean area, Asia, Africa and South America spring to mind and Micha's new station for India?
Looking at the archived data for stations participating in the detection of a stroke, there seems to be a large variability in the quality of the signals sent, not only between greens, reds, and blues, but even between stations of the same groups.
Should there be some measure of the "quality of signal" sent to the server? Signal to noise ratios? Index of Distortion?
I am sure that others may have better ideas, I shall sleep on this and maybe add more tomorrow.
Kindest regards,
Brian.
RE: Efficiency - pasense - 2017-10-27
This thread mixes several problems, I will try to seperate them:
- how to I optimize the S/N ration of a station?
- how do I measure the efficiency of a station?
- How do we handle stations "inside" the network and isolated stations?
My comments are:
For 1.): assuming the station doesn't have internal noise (suitable power supply, shielded/ unshielded cables, well placed antennas) only external noise counts. If you cannot identify and eliminate noise sources, there are only two options: if the frequency of the noise sources is higher than say 15 kHz use the LP filters and set the cutoff frequency accordingly; if not, lower the amplification and/ or raise the threshold.
For 2.) Egons suggestions are good, but what about a quality measure for the signal used in calculations? For stations within the network (but ONLY for them), this appears to be a good indicator:
https://www.meteomelin.be/staticfiles/bl...egion=1&id=
In the case of the few stations contributing to more than one region the regions should be combined in the efficieny measure, but if the regions are going to disappear anyway, this may not be required.
For 3.) Stations within the network (surrounded on all sides by other stations within a reasonable distance) need to be treated differently from stations at the fringe of the network like coastal stations or even completely isolated stations (on islands, isolated towns...) Maybe one could calculate a measure that indicates the integration or isolation of a station?
From the experiency of the other network that operates globaly (http://wwlln.net/) I would suggest to install the LN filters directly on the board since soldering the tiny ICs afterwards is really difficult. Wwlln uses a cutoff of 22 kHz since most of the sferic signal strength is below this frequency. If bought in quantity the cost of the LP filters is acceptable and the added flexibility in handling noise and improving S/N ratio is well worth it.
RE: Efficiency - readbueno - 2017-10-27
Hi All, We need to define what we each mean by "efficiency" and agree on a common definition?
In the case of:-
https://www.meteomelin.be/staticfiles/bl...egion=1&id=
Would it be possible to develop this tool slightly more and have an average accumulated total for various longer periods, updating every five minutes and only giving the results for the last hour, whilst very useful, does not go far enough. One day, One week, One month?
This need not be "real-time" for the historic data and would show the affects over time, especially if several graphs could be seen on the same screen at the same time for comparison.
This "pattern" might also help with the station stats pages, i.e. more variable periods of time rather than just the last 24 hours, or the total stored data since the station started.
This would help in "optimising" our stations and reduce the overall load on the servers.
A better explanation of how the auto functions work, some basic instructions and improving their working, would make it easier for people who do not have a lot of time to fully monitor and manage their stations?
Information and education should be part of running a station, if we don't know what we are doing, or what the overall aim of the network is, then different stations are going to have different goals and this will lead to less network efficiency.
Maybe the LP filters could also be managed by the server to allow for wider bandwidth at times of low noise?
I know that I have maybe drifted off the main topic of "Efficiency," but surely these and some of the other suggestions made in these recent posts are inter-related.
Brian.
RE: Efficiency - Breitling - 2017-10-28
(2017-10-27, 09:15)readbueno Wrote: Hi All, We need to define what we each mean by "efficiency" and agree on a common definition?
In the case of:-
https://www.meteomelin.be/staticfiles/bl...egion=1&id=
Nice graph. I would add some reference lines for a better interpretation:
RE: Efficiency - readbueno - 2017-10-28
Hi Luis, Yes I like that Idea. No-one is going to get near 100%. As we see, most stations are between your yellow and the black lines, that seems to be the "reasonable" area, although that should not discount stations in that area trying to improve!
The area to the right of the black line should be an indication that these stations are not working as well as they should/could, and that an effort should be made to improve this situation.
The nearer you get to the bottom right hand corner the more opportunities for improvement are open to you?
These stations are your Blitzortung neighbours, you should help, them just as you do your physical neighbours, when they need help.
Maybe the actual points could be made different colours to show the "health status" of the stations?
Brian.
RE: Efficiency - Breitling - 2017-10-28
(2017-10-28, 08:52)readbueno Wrote: Maybe the actual points could be made different colours to show the "health status" of the stations?
Maybe, or showing "online time percentage" if it is possible to read automatically.
RE: Efficiency - cutty - 2017-10-28
(2017-10-26, 21:03)Egon Wrote: Hi Folks,Just brainstorming:
I would like to change the measure for the efficiency of the stations. It may not be appropriate to calculate only one numerical value, but rather to introduce different possibly even competing measures for the efficiency of the systems.
The first value could indicate how many of the transmitted signals are involved in the calculated strikes. The second value would be distance dependent and indicates how many impacts with a certain distance to the own detector the station was involved.
The current combination of these values (thate what we are doing now) seems to be not usefull for adjusting a detektor.
Any ideas for further efficiency measures?
/Egon
Suppose we explore this, for each unique station, as opposed to, or in addition to, station's relative "Network / Region" performance. This suggestion is for 'Typical" installations. It's my belief that a second group might be considered, for example, those stat1ons that are 'purposefully' employed across regions or oceans. However they, I believe, should be
"qualified" on Efficiency and Effectivity overall before being placed into that group. And 'disqualified' if performance degrades. If makes little sense to me to have a station with a noisy environment attempting to serve 'cross region' or ocean, for example.
We define:
Goal: detect maximum number of strokes accurately with minimum number of signals
Efficiency: Ability to accomplish something with the least amount of time and effort.
Effectivity: Actual production of the intended result.
Efficiency = Strokes Detected / Signals Sent
Effectivity = Strokes Detected / Signals Sent - Strokes Detected
examples:
Station: 200 strokes with 1000 signals sent:
Efficiency = 200/1000=20%
Effectivity = 200 / (1000-200) 800 = 25%
Station: 200 stokes with 500 signals sent:
Efficiency = 200/500=40%
Effectivity = 200 / (500-200) 300=67%
Station: 800 stokes with 2500 signals sent:
Efficiency = 800/2500=32%
Effectivity = 800 / (2500-800) 1700=47%
That is, a station which sends a lot of signals, but few strikes is not very efficient.
For Effectivity; some of the 'signals' sent ARE strikes, and they are removed from the Total Signals, then strikes compared to 'remaining' (unused" signals.
Mike
RE: Efficiency - readbueno - 2017-10-28
Hi Cutty, How does this look in your scheme of things. I am the third black dot a little to the right, yours is the top black dot and another Mike is just a little to the left of my dot.
Whilst I feel that I have, even yet, considerable "headroom" for improvement, my station appears to work quite efficiently when there is Lightning to report and sends very few signals when there is not.
I presume that some of those signals at least are "Housekeeping" between the station and the server? Rather than just wastage.
As a European coastal station added to Region 3, I feel that the benefit to both regions when there are storms in the North Atlantic are significant.
Maybe more coastal stations would consider this option in various situations?
I am also considering, some time over the winter, to have a dedicated antenna, with which to assist, if possible, the new stations in Africa.
As nearly three-quarters of my horizon is ocean it should be possible to "null" out most of Europe by careful positioning of the antenna, I was thinking of a single three turn Mobius type, and for this specific example, a certain amount of Dxing would be legitimate and beneficial to the development of the Network.
This use of a deeply nulled single antenna could be of use in various other situations as well?
Regards.
Brian.
RE: Efficiency - pasense - 2017-10-28
We need to look at the signal/ noise ratio from stations first. Noise here means signals not from real strokes.
I just used the strikes/ signals plot and then the stations list to check the signals from some of the stations in the right bottom area, where the stations are concentrated that have low "efficiency".
I have not made a complete investigation, but a high percentage of them (but not all) clearly has a serious noise problem. Often this is noise with a frequency larger than 25 kHz. A lot of the signals are so terrible that I doubt that the owner of the station has looked much at them, or if he has, he is not able to interpret what he sees. Obviously, these stations are not contributing to the network. Identifying these stations has to be done by hand so it would take some time, but then the owners could be contacted and advised to improve the performance. If they don't, their signals could be ignored directly.
Another conclusion I would draw: increasing the number of stations without attacking this problem at the root is not advisable.