Blitzortung.org Forum

Station: 2 stokes with 2 signals sent:
Efficiency = 2/2=100%
Effectivity = 2 / (2-2) 0=???%

Hi All, Help, advice and education must be the Key here. I certainly don't want to see stations arbitrarily ignored if their station is not up to scratch.
Getting in touch with these people and letting them know that there is a problem with their station and that advice is available to help them resolve it if they can't work it out for themselves.
This is a hobby and most people will take good care to make sure that their station is doing the best that it can.
We all know, that sometimes real-life means that hobbies do not always get the attention we would like to give them!
Maybe better advertising of the Buddy system and people pairing up to keep the enthusiasm going would help?
Unfortunately not many of these stations post in the forums asking for this help, therefore they will have to be sought by other means initially.
Regards,

Brian.

Station: 2 stokes with 2 signals sent:
Efficiency = 2/2=100%
Effectivity = 2 / (2-2) 0=???%


This is why we need a better Algorithm? Make some useful suggestions!

Well. My overall belief is 'one stroke' located accurately with 1001 signals is better than zero strokes "detected" per 1000 signals sent...  in order to have detected that 'one stroke' the station sent 1001 signals... but became the final 'locator' required to register a stroke...while the other 'locators' only sent 500 signals ---while its 'efficiency' / 'effectivity was nil, especially if compared to the other locators,  the benefit to the network was 'effective'... it enable the network to achieve the network goal, with a whole lot of effort on the stations part, and that critical station show's lousy efficiency / effectivity 'uniquely' . Conversely,  network 'effectivity' / 'efficiency' is not necessarily related to a stations capability or performance as a unique station.
Also consider that some percentage of virtually any stations' signals ARE skywave reflected strokes, but timing and distortion of signals, and lack of reception by enough other stations, does not result in it being 'recognized' and is scores as a signal, but not a stroke... thus lowering any efficiency / effectivity.

If I choose to operate at a short range only, I can be highly effective, and efficient, as far as my "Goal".. see my above post... my goal being most efficient and effective at <500km for example.  But related to 'network' I will almost always not contribute as many strikes... especially if compared to 'network' totals.  If considered 'uniquely', in other words, 'how am I doing for my environment, established goal, etc'... my 'long term', seasonally, situationally, geographically, has a better chance of contributing better 'quality' data to the network, rather than a high 'quantity'.

I send a LOT more useless signals than I'd prefer.... it's an environment issue. I can back those down by decreasing range, except when a couple of near disturbers flare up that are very strong, and the burst repetition timing does not trigger 'burst mode' interference... I'd prefer to go interference, not send signals, as the design intent... these disturbers don't make critera for interference modes . Therefore my network contributions on either system are nothing  like their potential. But, having establishing both 'normal' and 'situational' operating goals, I consider my two stations quire effective and efficient, all things considered. Its NOT the equipment... its the situation and my established paradigms. 

For me to compare my stations with stations in a super quiet environment, such as Brian and Mike encounter, is absurd. I can 'never' match them, except in rare situations.  Now Three years ago, before the environment degraded, I'd go 'head to head' with anyone... 

And I believe that stations that attempt to 'match' other stations 'performance', be 'on top' of the 'effectivity' tables may well be chasing a dream, which is fine... after some time, I think a lot of us come to realize that this is a "Team" effort, and we put our stations in a position of performing as well as possible, with maximum 'unique' efficiency...

BRIAN, the 'communication' between server and controller is NOT a 'triggered signal'.
PASENSE, I totally agree with you.

(2017-10-28, 16:12)readbueno Wrote: [ -> ]Station: 2 stokes with 2 signals sent:
Efficiency = 2/2=100%
Effectivity = 2 / (2-2) 0=???%


This is why we need a better Algorithm? Make some useful suggestions!

Effectivity would be infinity.  In our case we establish an upper limit of 100

Dear Cutty, Of course you are correct, there is a happy medium.
My station Square lights up with only two or three others and sometimes there are not enough stations to register a stroke, maybe it is not the right sort of lightning. Who Knows?
Certainly for very near storms or heavy interference it is best for any station to stop sending to the server, that is the benefit of the network, most of those strokes are still going to be recorded by other stations, a thousand to one ratio seems reasonable under certain conditions, 100,000:1 is maybe less acceptable?
Comparing apples and oranges is not easy and most of us are trying to do our best.
I think what Egon is trying to find is a reasonable middle way and if that means excluding stations outside of certain limitations, so be it.
What you and the older stations have learnt has lead to the better development of the stations, over the years, greens to reds to blues to xyzs.
There will always be improvements and compromises possible, but we can't just sit on our laurels (or whatever!) and say that's it we are finished.
There is some way to go yet?

Brian.

(2017-10-28, 08:24)Breitling Wrote: [ -> ]

(2017-10-27, 09:15)readbueno Wrote: [ -> ]Hi All, We need to define what we each mean by "efficiency" and agree on a common definition?

In the case of:-

https://www.meteomelin.be/staticfiles/bl...egion=1&id=

Nice graph. I would add some reference lines for a better interpretation:

Your graph with the diagonal lines is GREAT !! I am always holding a straightedge up to my monitor. In some cases I have saved screenshots and added the diagonal lines with GIMP (a free OSS Photoshop clone). They should be added to that webpage.

(2017-10-28, 16:06)readbueno Wrote: [ -> ]Hi All, Help, advice and education must be the Key here. I certainly don't want to see stations arbitrarily ignored if their station is not up to scratch.
Getting in touch with these people and letting them know that there is a problem with their station and that advice is available to help them resolve it if they can't work it out for themselves.
This is a hobby and most people will take good care to make sure that their station is doing the best that it can.
We all know, that sometimes real-life means that hobbies do not always get the attention we would like to give them!
Maybe better advertising of the Buddy system and people pairing up to keep the enthusiasm going would help?
Unfortunately not many of these stations post in the forums asking for this help, therefore they will have to be sought by other means initially.
Regards,

Brian.

Indeed! Some stations need to be contacted and help offered to them.

Cutty, You should not be so modest.  Your instructions have been of great assistance to me whilst setting up my station.
Yes, I have taken the time to read most of the forums and I do monitor other stations for comparison with what I am seeing on the screen of my own station.
I am still improving, slowly.
Both your Red and your Blue stations seem to be always fairly near what most would consider about the optimum on the graphs!
I.e. You have taken the time and made the effort to find your station's Sweet spot.
The out-lyers on the graph, appear to be the problem and there are not many that could be accused of too little gain or too high a threshold, most of the stations that we are referring to, would err on the side of "high gains, low thresholds" or have a serious source of interference local to their station.
Auto, or semi-auto does not work at present, if your station is not reasonably well set up to start with?
This accounts, overall, to quite a small percentage of stations, but some of these stations send more signals per hour, than the rest of the Network combined.
I am not "trying to teach my Grand-Mother how to suck eggs!" but Egon did ask our for our opinions and suggestions, and it seems only polite, to respond with our views of how things are from a personal perspective. I am sure if I was in the "centre" of the network, I might be trying to defend a different view!
Discussion in the forums is very stimulating, but how many operators actually read the forums and take note of the suggestions made there in.
Some of the less well managed stations appear to be in "Institutions," where perhaps, they are not the responsibility of an individual.
"What is the work for all, becomes the work of no one?"
You only have to look at the stats to see glaring examples of this!
May be this is a different issue, but it appears to be at least part of the same problem, and needs to be addressed?

Brian.

ok, I will try to sum up some things i heard:

---

There are different criteria for optimization for different locations. So if we want a station to be optimized for certain criteria, then we need a separate value for that. This location specific value should be high when the station is optimized for it's position in the network.
That means for stations at the border of the network:

1. They should send many signals, even if only a few more strikes are detected.This helps to get the valuable new strikes. So (detected strikes) / (detected signals) should be low. Sounds strange, but that might be helpful.
   
2. In this area we do not know how many strikes are there actually and thus calculating the ratio of (detected strikes) / (actual strikes) makes no sense. It will be always close to 100%.
   
3. Stations at the border might be those, where let's say, there are not more than the magic 11 stations within 3000km.
   

... and for stations in the center of the network:

1. They should send NOT too many signals. So (detected strikes) / (detected signals) should be closer to 1. Here the graph proposed a lot of times is looking very promising. If a station is too far away regarding this, then we can assume it will not help the network at all.
   
2. As we know nearly every actual strike, here (detected strikes) / (actual strikes) makes a lot of sense. (actual strikes) should of course only be the strikes within the expected area, not all worldwide.
   
3. Stations like this have let's say more than 11 other stations within 500km.
   

This value, as we can see above, has completely contradictory optimization strategies. It is therefore usable to measure the contribution to the network, but not for the local optimizations. And the owner of each station should be aware of what to optimize for.

---

For the local optimization the criteria are different. We all cannot change the position of our detector, but what we want to achive is a good signal to noise ratio. So we need some help to find the right amplification, threshold, filter frequency or quality of the power supply. For that we should get help e.g. like this:

1. When there are repeated signals normally we have noise, even if the numbers are below the interference. That means that (sporadic signals) / (all signals) would be a good ratio. So we need to measure the sporadic signals. The first attempt that is already built into a receiver is the interference detection, but I think we can get better data there. We can measure the time difference delta_t between the signals and see if delta_t is constant several times. Also (2 x delta_t) should be detected as repeated. All those with a completely different delta_t should be sporadic. I know this requires a bit more calculation effort, but if we find a proper algorithm, then this would be a very valuable input for the auto mode.
   
2. Wheneven a signal has a lot of amplitude BEFORE the trigger point, it is very likely to be noise as well. Looking at the "bad" stations (I'm allowed to say bad because mine is one of them sometimes), I see a lot of signals like that SOMETIMES. So (bad signals with noise before trigger) / (all signals) should be low. It would be a good input for the auto mode as well.
   
3. We could measure the V_in several times in the background. If it changes with high slopes, then something could be wrong with the power supply. That was the case for me.
   
4. ONLY if a station has a lot of noise all day, then a manual change makes sense. So we should not punish someone if we don't know it's a permanent problem.
   
5. And we should mark the TEST SYSTEMs in all the graphs. When looking at some of the "real bad ones", I also found a couple of test systems, which makes sense but is not obvious in the graphs. And stations that are not maintained actively anymore (let's say the user did not change anything or login within the last 1 year) could also be excluded (not marked because we don't want to blame someone), at least if they are not performing at all.
   

All in all, most of the points are too much changing over time and depending on CURRENT noise, so it makes more sense to improve the auto mode (or manual mode with partly auto) to solve them. It also makes no sense to put everything into ONE number. Giving the numbers I proposed above could help to get an overview but is not the solution. And I guess that users, that are not ABLE to optimize their station right now, will only slightly improve. When I had the problem with the power supply, I nearly gave up. It does not help to give them bad numbers, only contructive advices help. So go for a better auto mode to solve the problems with the settings and go for real advices, e.g. also by mail, for proposed hardware changes (shielding, relocation, power supply, ...).

All interesting suggestions.  A question though on how the system puts together the signals to come up with a figure.  I too check on settings and what channels are being sent to the servers.  Now pardon a "newbie" question re efficiency I some times see current high efficiency with stations that appear to be only sending 1 channel.  Is that what happens with the system???? I would think that at a minimum a station should be sending channels 1A & 1B H fields to rate an efficiency, e fields channels a bonus.  My thoughts on the basics of this.

(2017-10-29, 10:17)orion_jb2001 Wrote: [ -> ]All interesting suggestions.  A question though on how the system puts together the signals to come up with a figure.  I too check on settings and what channels are being sent to the servers.  Now pardon a "newbie" question re efficiency I some times see current high efficiency with stations that appear to be only sending 1 channel.  Is that what happens with the system???? I would think that at a minimum a station should be sending channels 1A & 1B H fields to rate an efficiency, e fields channels a bonus.  My thoughts on the basics of this.

For some stations, setting up the channels one at a time, is a better solution than trying to sort out the mish-mashs of signals on the graphs, when several channels are operating?
If stations are still sending good and useful signals at these times, then they should still be entitled to whatever result the server gives them?
Limiting the results by imposing false conditions may be contra-indicative of what is needed, i.e. if a station has good signals on certain channels and not so good on other channels, it appears to be a reasonable idea to self restrict which channels are sent, until the problems could be resolved.
Obviously sending on three channels, or more, if your situation allows is the optimum, but as we know, not all stations are in this good situation and we should accept some variations for the best of what these stations can do.

Brian.

(2017-10-26, 21:03)Egon Wrote: [ -> ]I would like to change the measure for the efficiency of the stations.

That is good to hear, as you say the current measures are not overly useful in setting up a station. To get towards the top of the list they tend to push stations into sending lots of signals.

Part of this is the current distance values used (S up to 50 km, M up to 500 km and L up to 5,000 km) and the default sort being on L. Could the participation data be analyised against distance to produce an "optimum" detection distance? Then use that +/-, say, 500 km as the "M" measurement. S then going up to the lower M limit and L above the upper M lmit. The default sort being made on the M value.

The other problem is the use of all strikes within a region and the regions being very big. If you created smaller regions you get boundaries and problems with stations close to those boundaries. Seems sensible to use strikes in each of the distance ranges from the station as described above.

I quite like Cutty's:

Efficiency = Strokes Detected / Signals Sent 
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

Being based purely on the numbers from a given station it removes the influence of the total region count which generally has the affect of pushing a station that sends few but high quality signals down the list.

Not sure how to combine those with a distance. "signals sent" doesn't have any distance information but does that matter? I don't think it does.

The Signals v Strikes graphic is an excellent visual and quick check for "how is my station doing"? As there isn't a line at whcih a station swtiches for "good" to "bad", I think a fixed shaded background color would make interpretation a little easier, green (upper/mid left) to red (lower right).

(2017-10-30, 21:06)allsorts Wrote: [ -> ]

(2017-10-26, 21:03)Egon Wrote: [ -> ]I would like to change the measure for the efficiency of the stations.

~~~

I quite like Cutty's:

Efficiency = Strokes Detected / Signals Sent 
Effectivity = Strokes Detected / Signals Sent - Strokes Detected

Being based purely on the numbers from a given station it removes the influence of the total region count which generally has the affect of pushing a station that sends few but high quality signals down the list.

Not sure how to combine those with a distance. "signals sent" doesn't have any distance information but does that matter? I don't think it does.
~~~

Actually, it does... in a 'thought experiment'... I just didn't include my thoughts and examples I played with, since the 'signals' cannot be 'predicated' , and for basic simplicity of the suggestion...

The thought experiment runs sort of like this:
Briefly, divide the stations according to 'intent'... say, for example, three groups.

Group ONE > operators who haven't bothered to define or optimize well.  Anything 'in'.,..  Anything 'Out'.

Group TWO >  Stations that have established a 'relatively' quiet operational invironment, normally function with a 'High" "Strokes to signals", and have optimized for 'distance'... would include many of the 'dual' region / fringe stations perhaps. Since they already operate with 'low' 'excess' signals. They can operate for example as the 'group greater than 1200 km'. Their signals would also include the signals of a nearby 'group three', of course.

Group THREE> Stations optimized for <1200 km... those that run many excess signals when using high gains and / or low thresholds, or with frequent 'sporadics' that could  be eliminated, or minimized with lower gains / and or higher thresholds.  They now optimize for reduced distance. Optimizing in this fashon also lessens the typical "excess signals' relative to 'detections within 1200 km'.

And you could have perhaps a smaller group, perhaps operating E field only, or C Horizontal only, operating at distances <600km or any other subdivision similar...

The decrease in 'out of distance' signals, with an optimizing within distance should allow the same similar formulas too be used... though a group 2 station might show a 'higher effieciency' or 'lower efficiency' than a station in group 3, or vice versa, the rules are the same, and the 'comparisons' similar, within the group.  not necessarily outside the group.

Both groups should be much more efficient and effective than MOST of the 'group ones'... and my 'gut feeling' is that watching numbers between groups TWO and THREE would actually show similar efficiency / effectivity numbers... IF the group TWOs had strokes <1200 km subtracted from both signals and strokes ,,,

Again, an operator shouldn't compare 'outside his Group" , ...
But... Especially on group TWOs... a station should establish, over time, that it BELONGS in group TWO !  To do so, they'd have to operate as a GROUP one until the "Higher Effeciency / Effectivity" numbers qualified them as a 'group TWO'... if a station degrades over some time frame, than they should be 'demoted' to group ONE, unless they opted to go for Group 3.

All stations would begin as group ONE, and then optimize for, and qualify for group TWO or THREE.  Actually, I think the server can do that automatically, since it already is capable of breaking strokes down by distance.d A 'quiet' group TWO station might not have as many 'signals' at high latitudes seasonally, but the 'noise signals' may remain essentially the same.... for a Group THREE, some adjustmen might need to be made seasonally, since 'theoretically' their 'local noise' soruces would remain at similar levels year round, while signal numbers rise and fall.
In those cases a comparison with 'degradation' over time of total 'signals' might be applied.

Cutty, I like your proposal, especially the distinction between local (G3) and long distance (G2) stations. One of the problems ist now to find out to what degree a station sends real sferics, even from very distant strokes or only spurious signals. So far, I look at the life signal from my station to see if they look fine, but this is of course not really an objective measure. Looking for a solution I just stumbled on the following:
http://wwlln.net/spectra.html
These plots show the sferics as thin vertical lines and the noise sources as horizontal ones. This gives a very good idea on the type of noise environment the station is working in. Do we have any means to generate plots like this for our stations??
I miss also other indicators. Currently, only a few percent of the signals my station sends are used in strokes by the network, but looking at the signals I am quite sure that almost all of them are from real sferics. To judge if I could reduce the number of signals I would need a histogram of the peak voltage of signal sent and another one of the peak voltage of signals used for strokes. Preferably this should be a long-time histogram, not only for the last hour. I could then decide if I could raise the threshold in order to concentrate more on the signals useable for the network. Without this information, its almost impossible to make a reasonable choice.
In short, I miss a number of indicators that would allow me to make a objective analysis of the station.

Hi Pasense, Nice graphs. Even if we do not have the ability to do this, then stations must learn about their local environment and make the necessary adjustments.
Using the tools available, the probes themselves or even a simple AM portable radio, most of the relevant problems can be mitigated.
Help is also available from more experienced stations, if people will ask by pm or email, and it need not be done in the public view.
Some people seem very reticent to ask simple questions, maybe, for fear of appearing stupid or silly. Whatever?
We must overcome this and do all that we can to help these stations without belittling or humiliating them.
It just takes time and vigilence/diligence and having taken on the responsibility for one of these stations, it should be considered part of the "self-training" to get it working as well as possible in each situation.
O.K. I have drifted off topic again, but I feel that this might be a significant part of the problem.

Brian. 

readbueno, it is much easier with a number of better indicators to analyse the performance of the station. So far, it takes a lot of time and many people simple do not have this. To condense the performance of the station down to two numbers is not easy, at least I am missing some more analysis tools.

(2017-10-31, 07:39)pasense Wrote: [ -> ]Cutty, I like your proposal, especially the distinction between local (G3) and long distance (G2) stations. One of the problems ist now to find out to what degree a station sends real sferics, even from very distant strokes or only spurious signals. So far, I look at the life signal from my station to see if they look fine, but this is of course not really an objective measure. Looking for a solution I just stumbled on the following:
http://wwlln.net/spectra.html
These plots show the sferics as thin vertical lines and the noise sources as horizontal ones. This gives a very good idea on the type of noise environment the station is working in. Do we have any means to generate plots like this for our stations??

The 'waterfall' spectrums are great... in a manner of speaking, the 'compression' hatch marks, when they're turned on, in effect perform an analogous presentation.... 'hashed' areas considered 'noise.

Let's not overly digest the suggestion... any 'signal' entering the system, and hits trigger threshold will be a 'signal sent'. Now, a large portion of these are, in fact, a sferic. For Fringe area stations, at distance, for example, they may NOT be recognized by enough other stations to be called a 'strike'.  So, they degrade the 'efficiency' . 'effectivity' percentage.  Now, as other stations come online, or optimize better, the same sferic, previously just 'noise' can be recognized as a 'stroke'. Remember takes 4 stations minimum to locate. And some geographic regions with high density of stations require more stations minimum... hopefully improving location error (deviation).

Enough stations can improve everyone's 'efficiency / effectivity' to some degree.  Enough stations would potentially also enable the 'timing' of signals to be 'tightened' perhaps, thus making a dramatic improvement in deviation.

Pasense, I agree, but the initial instructions state quite clearly that this is not Plug-n-play. Some experimentation is going to be needed to get the best out of each situation.
Cutty, In Europe this validation count is much larger than four, I have seen it vary between 11 and 13. Even in Africa, with only Two mainland stations I believe the validation is 8.
In the case of Pasense for example, there are others, but he is in this conversation, His station has many very clean strokes being received, but there are not enough stations in position to validate these strokes.
This is the opposite of the effect in densely populated areas where several hundred stations may receive the same stroke and only a few are actually needed to validate its position.
I think this is where your grouping idea could be developed, these less populated areas need less strict criteria for validation, at least until more stations are located in suitable areas to make this less of a problem?
I can think of at least half a dozen stations that this is most critical, and if proven stations were admitted to this group on a performance basis, then it might improve the situation in these areas?

Brian.

Cutty, would it not be reasonable to discuss which tools are missing to enable the improvement of the station's perrformance?