Firing Accuracy Testing
This page is a scratch page re: testing how Firing Accuracy works. Results will be summarized back to relevant pages when done. Page initiated by User:MikeTheRed, but anyone is welcome to jump in. This is an informal page.
Notes & Nomenclature:
- FAT is short for Firing Accuracy Testing. Sorry... it just worked out that way!
- FFA is Final Firing Accuracy - the quoted percent seen when about to fire a weapon. Otherwise, there is soldier FA and weapon FA, plus other factors as seen in Accuracy formula and Firing Accuracy. For this work, only soldier and weapon accuracy have been hacked/tested, unless otherwise noted.
- Unless otherwise noted, all testing used:
- Autofire
- Mutons
- A "direct" line of fire (i.e., not a diagonal shot)
- A clear field (no obstacles)
- Same level (both soldier and Muton on ground)
Background
There's been a lot of conjecture about Firing Accuracy on the wiki and in X-COM forums. I couldn't recall much actual testing, so here we go.
- Soldiers can have FA from 40 to 125.
- Common weapons have the following FA:
Weapon Auto Snap Aimed Heavy Plasma 50 75 110 Std. Rifle 35 60 110 Laser Rifle 46 65 100 Std. Pistol -- 60 78 Laser Pistol 28 40 68
If you want to hack files yourself, here are bytes of interest. But also see BombBloke's Map Editor, which makes a lot of this cake.
"Max" means a large number; "z" (122) is the largest single-keystroke value when using MS EDIT:
UNITREF.DAT, record length 124: Offset Description 23 Firing Accuracy. Decreased by injury, so heal all your soldiers. 12 Current TUs - 250 for soldiers, 0 for aliens 25 Base TUs - 99 for soldiers, 0 for aliens 35 Energy Recharge - Max all 15 Current Energy - Max all 27 Base Energy - Max all 45 Energy Usage - Max all 18-22 Current Armour, F/L/R/B/U - Max all so nobody gets hurt. 29-33 Base Armor - Max for all 63-68 Fatal Wounds - Heal everybody 81 FA (hits) counter - Make sure it's zero 57 Psi strength - Aliens to 0, soldiers maxxed 37 Psi skill - Same thing 13 Current health - Set to Base Health. (You are injured if Current Health < Base Health.) 26 Base health 17 Strength - If you want to carry a lot, eat yer spinach. OBPOS.DAT, reclen 16: 0 Object type (ref. To OBDATA Row # id) - Edit ammo quantity for all weapons you're using. It's a lot easier to just e.g. set all Pistol clips (Object Type 1) to 250 than to try to figure out who's holding what. 8 Ammo quantity - Set to 250 \GEODATA\OBDATA.DAT, reclen 54: Row 0 Pistol Row 1 Pistol clip Row 34 Heavy Plasma Row 35 Heavy Plasma clip 22 Average weapon strength - Set to 1 (no damage since armor is so high) 31 Ammo damage type. 0=AP, 3=Laser, 4=Plasma. FYI (shouldn't matter) 32-34 FA, Auto/Snap/Aimed. Here's what counts 35-37 Percent TUs, for the same. Set to 1. This compares against soldiers' Base TUs and is rounded down. 1 Percent of 99 rounded down would happen to be 0 TUs :) 38 Clip size - Set to 250. Might be irrelevant after setting OBPOS, shrug. 49 Ammo boolean - Keep it set for clips
Note that because OBDATA is in \GEODATA, you have to seperately put it in any pkzip savegame backups you might be using. Also note that it's reloaded each time you start a combat for DOS X-COM, but it might not be for WinCE (GEOSCAPE and TACTICAL are one .EXE for WinCE). Be that as it may... if you e.g. set Weapon Accuracy to 100 and leave it there, then soldier FA decides your FFA.
For most testing (unless trying to watch each shot), set your Fire Speed to 6. And if you're using DosBox with DOS X-COM: Crank your CPU cycles way up, 30k or higher (Control-F12). Bullets hit practically instantaneously. (Does the WinCE version have a similar thing?)
A savegame that incorporates most everything above is Media:FAccTest1.zip. All desert dunes have been levelled so you've got clear shooting with the four soldiers in upper right.
Test 1: Benchmark
I chose a middling FFA:
Soldier Accuracy 90% Weapon Accuracy 50% Final FA 45%
Here are the results for FFA 45%:
Observed Percent Hits Observed Calced %Miss Model Versus Dist Min Ave ± SDs Max Ss Shots Tot N Spread Spread =Hit % Hit Observed 1 98.8% 99.6% ± 0.6% 100.0% 4 250 1000 0.0 2 74.8% 80.2% ± 3.3% 86.0% 8 250 2000 0.9 1.0 68% 80.5% +0.3% 3 62.4% 68.0% ± 3.3% 72.8% 8 250 2000 1.3 1.5 45% 66.7% -1.3% 5 53.2% 58.6% ± 2.4% 61.2% 8 250 2000 3.0 2.5 27% 55.6% -3.0% 10 48.8% 51.3% ± 1.9% 53.8% 9 240 2160 5.0 5.0 14% 47.3% -4.0% 20 38.0% 44.2% ± 4.2% 48.8% 8 250 2000 11.0 10.0 7% 43.1% -1.1% 40 36.4% 41.5% ± 3.4% 46.8% 8 250 2000 17.0 20.0 3% 41.1% -0.4%
Where
- Dist is distance from soldier to Muton; 1 means, soldier is directly adjacent to target (Soldier is 0, Muton is 1 away).
- Stats are explanatory. SDs = Standard Deviation of sample
- Ss = Soldiers, each of whom made Shots shots, for a Total N as shown.
Example: For Distance 1, 1000 shots were made (4 soldiers x 250 ammo), of which 996 were hits (Unitref[81]). Thus, Average Percent Hits is 99.6%. The single soldier who missed the most (Min Hits) had 247 hits (247/250=98.8%). Etc. Min, Max, and SDs are included as estimates of the variability I saw (and you might, too). Average is always total hits divided by total shots. Then comes some model-related info, explained below:
- Observed Spread is how "wide" I saw the farthest misses miss the target by, to his side. In other words, if the Muton were a target on a firing-range wall, I saw misses as far as e.g. approx. 2.5 tiles to the side of the alien's center (a "radius"), which makes for a total "diameter" of 5 tiles centered on (and including) where the alien target was on the firing-range wall. Diameters are used instead of radii because they're easier in the math model.
- Calculated Spread is the linear model calculation estimating the observed spread.
- Percent Miss = Hit is, as per the model below, if 100%-FFA% of shots are "expected" to miss, but the target takes up X percent of the firing-range wall, how much of those equally-distributed misses then become "accidental" hits.
- Model Percent Hit is the total of pre-determined hit percent plus accidental hit percent. (It's not e.g. 45% plus 68% for distance 2 because only 55% of shots are missed; only 68% of 55% adds to 45%. But I get ahead of myself; read model details.)
- Versus Observed is Model Percent Hit minus Observed Average Percent Hits.
As can be seen, this Final Firing Accuracy of 45% is in fact only 45% at a distance of approx. 20.
Digression re: Final Firing Accuracy as quoted in the game
It's clear that the FFA seen in the game (the pop-up when preparing to fire a weapon) is some sort of general statistic. It can't possibly apply to actual situations. Consider that you might be faced by five aliens. The dudes are at greatly differing distances, with some half-blocked, others not. Or maybe even, there isn't a single target to be hit. What are you quoted? The same old FFA, regardless of anything going on. Or not going on.
Final Firing Accuracy has always been found to follow the Accuracy formula - a formula which includes zero target information.
Although FFA is devoid of target information, of course it had to have in-game relevancy. And distance is an important part. What's the most relevant distance in a game? Very near? Very far? At the moment, I lean toward FFA being the likeihood of hitting an average-sized alien at distance 20 on an open field (outdoors, no blockage).
So. FFA is a generalized number. Never intended to show specific FA versus a target. As work progresses, I/we hope to pin down what the programmers considered a/the commonly-encountered game situation.
Test 1: Model
Due to past experience, the curve followed by FFA 45% versus distance looks like two different functions to me: a constant due to FFA and a variable due to missed shots.
If 45% of shots are "guaranteed" to hit at FFA 45, what happens to the 55% of shots that miss? In a simple model - and we all know X-COM likes to be simple - misses might be evenly distributed across a "miss space". In the instance of a game like this, a "simple miss space" could easily be defined as an angle of spread of bullets that miss. What happens if you're very close to the target? In a simple model, they aren't "forced" to be misses... they hit because the target is right in front of your sprayed misses. But if you are far away, your target has a much smaller silhouette relative to everywhere your spray might go. And at extremely far distances, the observed hit percent should approach your theoretical "guaranteed" hit percent.
Keep in mind that FFA as quoted in the game - 45%, in this case - was never a guaranteed hit percent, itself. As discussed above, it's probably a "common situation" hit percent - and it's not common to be extremely far from your target. Then compare the curve seen in the graph above. This is a long way of saying that there may be a "guaranteed" hit percent, but in reality it's probably not the Final Firing Accuracy quoted in the game. And it's probably not 45% for my FFA 45 tests. It's probably less than that; 45% is probably what's usually seen in the common situation that I conjecture is Distance 20 on an open field.
Initially, I modelled FFA versus distance as having two components: 1) a guaranteed hit percent of 45%, and 2) 55% of misses, which "accidentally" hit your target as a function of distance from the target. "Accidental" hits, in my simple model, got very simple mathematical modelling. Specifically, they hit the target accidentally as a function of proximity. When very close, almost every "miss" actually hits. At great distance, very few misses do.
I found that I could model the FFA 45 curve fairly well using an equation that considered the absolute accuracy for guaranteed hits to be 39% - you would hit the alien when infinitely far away, 39% of the time. Past that, the 61% of misses "accidentally" hit the target more, as you get closer. Specifically, the simple model had the Muton as being 0.68 tiles "wide", and the angle of spray of the misses was such that, for every additional tile distant from the Muton, the spray of bullets was increased by 0.5 tiles at the level of the Muton. Imagine the Muton as being a target on a firing range wall: If you are shooting at him from Distance 6, there will be a spray of misses 3.0 (6x0.5) tiles wide, centered on the middle of the Muton. So, bullet holes would be seen up to the far (distal) edge of the tiles to the immediate left and right, of this paper Muton target on a firing range wall.
The results of this simple model are shown on the graph above as the dotted line "Model Ave Hits". As you can see, it does very well when very close and very far from the target, but it underestimates the number of hits at mid-range distances (worst case, Distance 10). This was after tweaking the numbers involved, specifically, the guestimated Muton's width, and the "true" FFA of the gun. The spread of bullets was not tweaked, since it seemed fairly clear that it was that value, in the game. Conversely, Muton width and true FFA are just guesses.
This very simple model has an obvious drawback: It treats the data as though bullets can only be in one X,Y plane. In other words, you only ever hit a horizontal line drawn on the firing range wall. In the real game, though, bullets obviously also have a Z component - they can also miss by going too high or too low, instead of only too far left or right. Be that as it may, this simple conceptual model did a pretty decent job of handling the results, without resorting to any major fudge factors. So it may be onto something.
Test 2: Ceiling for zero misses
In previous work, Ethereal Cereal showed me how ceilings (and floors) are important. There might be a lot of variability in between these limits, but if you can pin down the limits, you learn how far a function can be pushed. This may (or may not) ultimately be important information for understanding how Firing Accuracy works.
I used the same distances as in Test 1, and found the following results for the FFAs needed to never miss, at each distance:
Dist FFA N Est
1 66 3250 100
2 100 1000 100
3 101 1000 101
5 101 750 101
10 102 1000 102
20 106 1000 104
40 108 1000 108
Estimate is the estimated FFA required for 100% hits, based on the simple equation:
FFA for 100% hits = INT(100 + (Distance+2)/5 )
Or, in other words, the required FFA is 100 at Distance 2, 101 at Distance 3, and otherwise increases by 1 for every 5 tiles farther from the target. As can be seen for Est, it did fine for all points but Distance 20. I did get some hits for FFA 104 and 105 at Distance 20, so I don't know just what's going on there, shrug. Anyway, this function otherwise seems fairly solid. That's all I'm going to do with it for now.
Distance 1 (right next to the target) was a big exception. It was also a huge hassle to test, because misses were very rare... I walked FFA all the way up from 45 to 66, a couple of times needing 800 shots before one missed. That's why I tested it so much more than the others distances; for other distances, misses were usually seen very quickly (100 shots or less). I'm not sure what to think about how it differs so much, but Distance 1 is strange, anyway. So there it is.
Test 3: Interchangeability of weapon and soldier Firing Accuracy
As a simple test, a soldier at distance 3 had, on the one hand, weapon accuracy (WA) 250 and soldier accuracy of 40 (SA; FFA 100%). And on the other hand was one with WA of 40 and SA of 250, just the reverse (but also FFA 100%).
Observed Percent Hits
WA/SA Min Ave ± SDs Max Ss Shots Tot N
250/40 98.8% 99.55% ± 0.33% 100% 8 250 2000
40/250 99.2% 99.55% ± 0.26% 100% 8 250 2000
100/100 99.6% 99.87% ± 0.23% 100% 3 250 750 Additional data from Distance 3 ceiling test, but I
stopped as soon as I saw a miss, so it's biased data.
As can be seen, there is virtually no difference. That's not a typo; both groups had exactly 1991/2000 (99.55%) hits. We usually suspect anything entirely equal as being bad info, but every now and then, it actually happens.
There may conceivably be things going on with other factors - such as Fatal Wounds - that might cause the FFA you see in the game, to not be the actual FFA that's used... they might not be simply multiplied as per the Accuracy formula, and affect shot results in additional ways past what can be seen with FFA. But I very highly doubt they are treated differently, based on how simple X-COM approaches things, in general. Anyway,
WA and SA seem entirely interchangeable relative to final FA. For these two important parameters, only FFA matters.
Digression: Why test with 100% FFA at Distance 3 when there are almost no misses?
The reason is that there is much less variability in results, when probabilities are either very high (approaching 100%) or very low (approaching 0%).
The expected variance of a binomial (yes/no, a.k.a. hit/miss) variable is N*p*(1-p), where p is the percent likelihood (percent hits, here). If you were to graph it, p*(1-p) is at its "worst" (has the most variance/variability) at .5 - right in the middle, when you expect 50% hits. At that point, p*(1-p) = .5 * .5 = .25.
Conversely, at the very edges (almost all hits or misses), variance is e.g. 0.01*0.99 = 0.0099. That's less than 1/25th of the variability at p=.5. Variance approaches 0 and finally reaches it when there are indeed all hits, or all misses.
In this specific instance (Weapon versus Soldier Accuracy), the results for each test (250/40 vs. 40/250) would undoubtedly have differed by a fair amount, if I chose a situation where there was approx. 50% chance of a hit. In other words, the results would have suggested that one of the two (WA or SA) had more "weight". This would've required more testing to make sure they had equal weight. But choosing a point with almost all hits or misses - such as how FFA 100 at Distance 3 is just barely below the 100% hits level - I could see whether there was a difference, with minimal testing.
Notes
Currently (11/26/06), ideas on FA testing are being discussed in BombBloke's StratCore thread on his great automation tools. Please drop by and jump in!