User:Pi Masta

Alright I finally created this page. Tired of seeing so much red on the Recent Changes page.

Um, I like computers and math (hence Pi) and of course XCOM or I wouldn't be here. Just something about the gameplay and concept of the game intrigues me.

PyXcom

I'm hoping to get a possible 'free-for-all' editor going in python. I had success getting an old version of Python to work in DOSBox, though it's not very stable. I'm working on a 'back-end' (currently dubbed PyXcom) for all the saved game files so applications would just have to worry about what values to put in and not formatting the file correctly, giving it a more 'pythonic' interface. Could still use work, but I have a simple class that can read and write .DAT files (really any fixed-length files), and is somewhat simple to add new fields as values in files are discovered.

From this I could make a Tk/Tcl version that would require a much later version of Python to do editing on modern machines (including *nix, and probably Mac). This would probably do more in depth editing, of say the bases, stats of objects or soldiers, etc.

Before that I think I'll make a DOS friendly version (aka command prompt, maybe keyboard selection) that could implement changes (I'm thinking of making a better soldier equip screen that runs before going into tactical, downside is I don't it will be easy to display images).

Another idea I have is to make a Monitoring program (PyXMon?) that can watch the save game and MISSDAT directories and apply patches to them automattically. The main use this would have is renaming soldiers since it is impossible to have an App, like XcomUtil,rename them after combat (GEOSCAPE updates the stats then), instead you could save your game, possibly wait a second or 2 and then reload it. The monitoring program will catch the new save game and apply the fix. This would also go with the 'Finished' Dirt Modules bug, though handling that between battles would probably be enough. Another application would be for Data Miners to have the scripts automattically output data from files (in a more readable format) on saves, for instance Zombie's initial Funds data could have had a script written in python to automatically append a file with the data about the countries.

I have too many ideas, and most likely I won't complete these projects, but I hope to get close. I'm updating this Wiki for the most part on the files, and trying to crack some of the others.

Current Status

I have a large number of files supported and I've started on making a layer on top of it. Namely BaseInfo, SoldierInfo, LocInfo, and CraftInfo that can all be connected together and I have made a 'master' module to do this given a directory for it to load from. Let me give some examples:

>>> t = Geoscape('missdat')
>>> t.base[0].name
'Main'

>>> for x in t.base[0].soldiers:
        print x
D Likhachev (r)aBRst (Rookie)
F Robinson (r)aRbr (Rookie)
Evans TU (C)C-AC*ST* (Captain)
.......
Sarah Watson (r)arbrst (Rookie)
Ed Kemp SACK (Rookie)

>>> for x in t.loc.used:
        print x, ':',  x.data	
Loc[0] - XCOM Base : Base 0: Main
Loc[1] - XCOM Ship : Skyranger-1
Loc[2] - XCOM Ship : Interceptor-1
Loc[3] - XCOM Ship : Interceptor-4
Loc[4] - XCOM Base : Base 1: Asia
Loc[5] - Crash Site : Large Scout-51
Loc[6] - XCOM Base : Base 2: America
Loc[7] - XCOM Ship : Interceptor-5
Loc[8] - XCOM Ship : Skyranger-2
Loc[10] - XCOM Ship : Interceptor-6
Loc[11] - XCOM Ship : Interceptor-7
Loc[12] - XCOM Base : Base 4: Australia
Loc[13] - XCOM Base : Base 5: South America
Loc[15] - Alien Base : None

>>> t.loc[5].data.damage #The crashed large scout
150

>>> for x in t.loc[1].data.soldiers: #Skyranger 1's soldiers
        print x, ';',	
D Likhachev (r)aBRst (Rookie) ; F Robinson (r)aRbr (Rookie) ; Evans TU (C)C-AC*ST* (Captain) ; Robinson TU (Co)ACrbr*ST* (Colonel);
C Dodge (S)*AC*Rbrst (Seargent) ; Yuzo Kojima SACK (Rookie) ; K Okamoto (r)W-ACr (Rookie) ; Yataka Shoji (r)C-st (Rookie) ; 
Austin King SACK (Rookie) ; Y Matsumara (r)C-W-ACR (Rookie) ;

And now to show that you can chain these together. This one get's the second LOC.DAT entry (which is skyranger-1), grabs its data (the skyranger craft itself), gets the craft's second soldier (remember is all 0-based), then his/her base, that base's location, that location's data (which just goes back to the base), that base's last soldier, and then that soldier's name.

>>> t.loc[1].data.soldiers[1].base.loc.data.crafts[0].soldiers[-1].name
'Y Matsumara (r)C-W-ACR'

This is all just the tip of the iceberg!

Right now I just have mostly Geoscape stuff made for use. The Tactical saves can be manipulated but there's no referencing other files or what not. So far there is no GUI. This is intentional as it is mean to run at the command prompt with XCOM if possible. I might whip up a generated GUI using tkinter, one that just makes fields based on the information from these class objects (read: not going to look pretty). But right now I think I'm going to get on to the Tactical files (the main ones, OBREF, UNITREF, etc).

Oh and I do have a stat string generator that is very customizeable, although probably not as simplistic.

As a tool

I've whipped up a handy file monitoring script that will run a function when a file is modified. With this I can have it automatically apply patches or fix issues whenever a game is saved. Also it could be possible with modification of the batch file that switches between Tactical and Geoscape to have it add or even replace parts of the game. (I'm trying to get a soldier editor, but there's a few issues I have to work out)

Mainly though, I can use this monitoring process to help figure out what certain bytes mean. For example when I was looking for what offset 12 does for OBPOS.DAT, I simply monitored one of my save folder's obpos.dat file. Whenever it changed my script would parse the information and compare it to the last known information (rather quickly too might I add), it then printed to a seperate console the results as well as appending it to a log file. I even added more functionality by having it read the SAVEINFO.DAT file to get the save name, thus I could put some comments on what I did while I saved. Put this all together and I could see what happened to the files (or what parts I want to know about) as often as I saved, no need to alt-tab out and run a script.

'GLOB' Files

I've noticed a lot of saved files end in 'GLOB', presumably short for Globe or Globals.

 BGLOB.DAT - ambient lighting on tactical maps
 IGLOB.DAT - date and time
LIGLOB.DAT - Money (I also think it holds income,expenses, etc for graphs as well)
UIGLOB.DAT - Largely unknown, but holds scores for completed research?
 WGLOB.DAT - Tactical turns elapsed, number of aliens/soldiers/tanks/civvies kills as well as score

LEASE.DAT

some working notes on lease.dat

fixed length of 180 bytes, prime factorization of 180:

2^2 * 3^2 * 5, 

factors of 180: (probable line widths)

2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 30, 36, 45, 60, 90, 180

only offsets that seem to change (out of about 14 files) (compared from 1st to 14 others, but not 2nd or 3rd etc with the other 14 so could be more)

0-1,  5-17,  20-23,  26-28,  32-35,  38-41,  44-47,  50-53,  56-59,  62-65,  68-71,  74-77,  80-83,  86-89,  
92-95,  98-101,  104-107,  110-113,  116-119,  122-125,  128-131,  134-137,  140-142,  146-149,  152-155,  
158-161,  164-167,  170-173,

There is a lot of symmetry in this format, lots of double 0's will line up at the line widths of the factors, with the exception of the very first byte, almost as if the first record is somehow different...

On the fields that change, there doesn't appear to be many different values for them, usually only 2 or 3, with the exception of the first byte again.

Patterns from 14 files

Out of all 14 files I'm looking at there are only 11 variations which make offsets 0 and 6 special as they are the only fields to have 11 different observed values. There are only a couple of offsets that didn't fall into 2 categories perfectly, this is demonstrated by the background of the offsets. Orange values are only seen with other orange values, and yellow values the same. Thus if say offset 0 is 65 (orange) all the bytes after offset 14 will be the ones highlighted in orange. If there is no background then it has been seen with both orange and yellow.

NOTE There's a good chance I goofed something up reguarding the orange and yellow's on offsets 1, 7, 12, and 13

offset	observed values											#	comments
0	65	37	234	108	143	177	181	22	215	61	53	11
1	0	1	10									3
5	77	141										2
6	32	197	71	234	235	145	51	24	56	186	138	11
7	0	1	2	10	6							5
8	13	200										2
9	196	4										2
10	94	0										2
11	248	0										2
12	120	208	90	180	14							5
13	0	1	2									3	Seems to be 0 for saves, 1 after combat and in missdat, 2 only for before combat
14	7	136										2
15	38	60										2
16	136	142										2
17	71	216										2
20	78	19										2
21	248	199										2
22	139	70										2
23	70	252										2
26	70	137										2
27	248	70										2
28	115	254										2
32	252	104										2
33	38	239										2
34	198	25										2
35	7	232										2
38	201	80										2
39	203	255										2
40	200	118										2
41	4	254										2
44	30	252										2
45	184	30										2
46	136	104										2
47	60	251										2
50	161	14										2
51	172	232										2
52	19	238										2
53	199	226										2
56	0	10										2
57	0	11										2
58	137	192										2
59	70	117										2
62	60	118										2
63	30	254										2
64	104	255										2
65	239	118										2
68	29	118										2
69	15	8										2
70	80	255										2
71	255	118										2
74	255	157										2
75	118	15										2
76	252	235										2
77	30	25										2
80	25	254										2
81	144	255										2
82	14	118										2
83	232	252										2
86	131	14										2
87	196	64										2
88	10	1										2
89	11	70										2
92	17	94										2
93	255	252										2
94	118	38										2
95	254	128										2
98	252	117										2
99	255	187										2
100	118	184										2
101	8	1										2
104	6	201										2
105	232	202										2
106	157	4										2
107	15	0										2
110	255	0										2
111	118	0										2
112	254	30										2
113	255	184										2
116	232	142										2
117	236	216										2
118	14	199										2
119	64	70										2
122	252	0										2
123	196	30										2
124	94	197										2
125	252	86										2
128	63	0										2
129	0	67										2
130	117	205										2
131	187	33										2
134	0	137										2
135	31	78										2
136	201	254										2
137	202	31										2
140	200	254										2
141	2	37										2
142	0	24										2
146	136	0										2
147	60	27										2
148	142	192										2
149	216	247										2
152	254	201										2
153	24	202										2
154	0	4										2
155	30	0										2
158	6	0										2
159	184	0										2
160	0	86										2
161	67	30										2
164	114	60										2
165	3	142										2
166	137	216										2
167	78	233										2
170	138	38										2
171	70	128										2
172	254	63										2
173	37	32										2
176	61	38										2
177	1	128										2
178	0	63										2
179	27	0										2

Offsets 1, 7, 12, 13

Comparing offsets 1, 7, 12, and 13 (the only offsets that will not 'determine' the values after offset 14 (and others)).

NOTE There's a good chance I goofed something up reguarding the orange and yellow's on offsets 1, 7, 12, and 13, most of it should be accurate though

1	7	12	13	Comments
0	0	14	1
0	0	180	0
0	1	90	0
0	10	90	0
0	10	120	0
0	2	120	0
1	2	180	0
10	6	14	1
10	6	208	2

Speculation

LEASE as the name of this file makes me think of it pertaining to either:

• Purchased craft monthly (shouldn't need this much data)
• Base monthly costs

The 'orange' set of values seems to be the most popular and 'simple'.