Talk:ASTORE.DAT
Just two observations I had while running a test on this file.
- While aliens are in transit, the game will stop you from removing the living quarters at the destination (assuming you're in a situation where you want to get rid of all your living quarters) even though all X-Com staff have vacated the base.
- A base that is dismantled (while aliens are still in transit) all the way down to the ground will not cause the aliens stored in astore.dat to be deleted. If the base is later rebuilt, the aliens will still be present.
- This was with deconstruction of the base. Must test again with base destruction.
- NKF
Windows version different
In the Windows version of the game, the structure of AStore.dat seems to be different. The file is a fixed size of 3600 bytes, and at a quick inspection, the format does not appear to match what's posted on the main page. I'll post more once I know it. --RobinHood70 14:56, 24 March 2008 (PDT)
TFTD astore.dat
The TFTD version of astore.dat (this is from an older version that only makes the 600 byte file) seems to have a lot of extra data in it.
From one of my TFTD games:
[ 0] Alien: Aquatoid(Squad Leader) Base: 0 Extra: 00 28 00 14 00 00 00 00 00 [ 1] Alien: Aquatoid(Soldier) Base: 0 Extra: 51 53 fc 8b 5e 10 c4 7e 00 [ 2] Alien: Aquatoid(Medic) Base: 0 Extra: 7e 0b c5 76 0a 8b 4e 0e 00 [ 3] Alien: Aquatoid(Medic) Base: 0 Extra: f0 5b 59 5f 5e 1f 07 c9 00 [ 4] Alien: Gill Man(Commander) Base: 0 Extra: c0 66 0f b7 db 66 0f b7 00 [ 5] Alien: Gill Man(Technician) Base: 0 Extra: d2 66 0f b7 ff 66 0f b7 00 [ 6] Alien: Gill Man(Soldier) Base: 0 Extra: 00 00 66 0f b7 46 06 0b 00 [ 7] Alien: Gill Man(Soldier) Base: 0 Extra: db 66 0f 02 c0 eb 00 66 00 [ 8] Alien: Gill Man(Squad Leader) Base: 0 Extra: ea 10 c9 cb c8 00 00 00 00 [ 9] Alien: Aquatoid(Medic) Base: 0 Extra: 06 0b c0 74 06 66 0f 03 00 [10] Alien: Deep One(Terrorist) Base: 0 Extra: 8b d0 66 c1 ea 10 c9 cb 00 [11] Alien: Deep One(Terrorist) Base: 0 Extra: 00 1e b8 88 3c 8e d8 c7 00 [12] Alien: Deep One(Terrorist) Base: 0 Extra: 8b 46 0a 89 46 ea 8b 46 00 [13] Alien: Gill Man(Squad Leader) Base: 0 Extra: 8a 46 06 b4 3d 89 46 fe 00
The "Extra" field is showing the bytes past the first three (type, rank, base) known fields. None of these aliens are, or have ever been, transferred.
The last field is generally 0, and I understand that one turns to 0xFF during transfers.
Could the extra data have something to do with location of capture or such? Or date and time? Or is it just random uninitialized garbage?
I'm almost suspecting the "random unitialized garbage" theory as the rest of the file looks like garbage:
... [ 45]: 00 fe eb 03 b8 ff ff 1f c9 ca 08 00 [ 46]: 00 08 00 00 1e b8 88 3c 8e d8 c4 00 [ 47]: 00 ff 46 06 26 8a 07 c4 5e 0a ff 00 [ 48]: 00 26 88 07 0a c0 75 ea 8b 46 fc 00 ...
Save for two fields (alien type and the 'transferring' flag), which are both 0. The original XCOM fields are cleaner, the unused ones are largely zero, save ones where an alien was researched, which definitely clears the first byte (but not the second one indicating rank)
Here's one from the original game, no transfers, although some have been researched:
[ 0] Alien: Floater(Medic) Base: 0 Extra: 00 28 10 14 00 00 00 00 00 [ 1] Alien: Floater(Medic) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [ 2] Alien: Muton(Navigator) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 (unused) [ 4] Alien: Floater(Engineer) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [ 5] Alien: Floater(Medic) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 (unused) (unused) [ 8] Alien: Floater(Soldier) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [ 9] Alien: Floater(Soldier) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [10] Alien: Floater(Engineer) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [11] Alien: Sectoid(Navigator) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [12] Alien: Sectoid(Leader) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [13] Alien: Floater(Navigator) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [14] Alien: Sectoid(Leader) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [15] Alien: Floater(Medic) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [16] Alien: Sectoid(Leader) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [17] Alien: Sectoid(Leader) Base: 0 Extra: 00 00 00 00 00 00 00 00 00 [18] Alien: Sectoid(Medic) Base: 0 Extra: 00 00 00 00 00 00 00 00 00
Most of those unknown fields are 0 in the original, although that 00 28 xx 14 pattern on the first record looks familiar from the first record of TFTD's file.
...
If that is garbage, is it possible that it's being pulled out of a "template" file in the executable from an incorrect address?
When the game runs, it assigns structs and arrays to certain positions in memory, but doesn't bother to clear the previous contents (at least, the DOS versions of the games certainly don't, though I think CE might). "Uninitialised bytes" always have a flag somewhere that indicates whether you're supposed to pay attention to them or not, but the games don't care about these flags when saving/loading - they just grab whatever's there and copy it byte for byte.
Hence if a certain area of assigned memory space never actually gets used, then yes, you'll get garbage data. This might belong to previous save games, or even entirely different applications - for eg, running DOS UFO under Windows 9x would often result in registery extracts, web history, etc getting embedded in your saves. A common place you can find such garbage data is in the padding space of fixed length null-terminated strings (eg soldier names).
Another point, even the CE versions of the games don't clean the previous save game files before writing to them. If you save in a slot that's been saved in before without manually deleting the previous save files first, then you WILL see data from multiple save games in the same files. A good example of this is MAP.DAT. - Bomb Bloke 21:46, 17 August 2010 (EDT)
That sounds like it's allocating from the heap -- given all of the fixed size structures in the game ( ASTORE[50] this, BASE[8] that, OBPOS[160] here, SOLDIER[250] there) I have been assuming up until now that these are static allocations, out of the BSS or Data areas, which are always initialized at runtime (BSS to zero, data to whatever you specified in the source). Generally it's best practice to statically allocate static structures (as implied by the name) as it reduces the risk of pointer screw-ups and reduces memory fragmentation and malloc's overhead.
If it's coming from the heap though, they should have been less stingy with the maximums.. 'specially on ASTORE, OBPOS, and such. I mean, the whole advantage of the heap is that it's NOT static.. :S
However, this wouldn't be the first time that I've seen less-than-optimal coding...
It would also have been nice if they had cleared the old junk out before saving (I noticed soldier.dat is like that: "Rene Buchard\0ski\0\0\0...") as it would make savegames compress better, for those with that doublespace thingy, or those of us zipping up savegames. The overhead from calloc and memset isn't quite so bad if it's handled intelligently. For example, they'd only have to do that once on startup for the entire soldier.dat array, and then just on an individual soldier marked as deceased at the time that they're removed from the game..
Hey, do you think they'll ever release the source on EU/TFTD? That never hurt any of the other companies that did it on legacy products (Homeworld, Quake, etc)
The copyrights have been passed all over the place, but not the code - 2k doesn't actually have the source for any of the games.
You'd expect some of the original developers to have it - if memory serves, at least one has mentioned they'd be happy to give up the goods if granted permission by 2k. All attempts at setting that in motion have failed thus far, though. :(
- Bomb Bloke 20:00, 18 August 2010 (EDT)