Talk:Research

I'd like to see 'optimal research trees' being discussed, with the obvious endpoints of 1) Avenger (which includes Firestorm), 2) Flying Suit, 3) Heavy Plasma/Clip, 4) Plasma Beam, 5) Psi Lab/Psi-Amp. Working in the advancing profitability of Motion Scanner, etc., ultimately to Laser Cannon. Surely there are opportunities for overlap and optimization, depending on one's goals (quick victory, best Avenger/Psi fighting force)... discussion / new wiki topic? I'm new to wikis - pardons if questions aren't usually asked this way?

---

Good job on the re-write, Ethereal. It was another eclectic collection page that you've made much more neat and informative.

I see you hacked RESEARCH.DAT - excellent work! We really needed those. Was there only the project name and research times in RESEARCH?

Now that I know all the projects are there, maybe I'll revisit those total research time findings. If you want the spreadsheet I used, you're welcome to it. Ultimately though it's more a factoid than anything important.

Were those Notes in the Research Trees fixed correctly? (Notes 1 & 2 were incorrect and meant the same thing?) Also the current lone note is slightly hard to find... Maybe the "notes" should simply be noted in italic, right at the point where they apply?

FWIW the Discussion note (above this) is one of the very first things I ever put on the wiki. I didn't know how to sign then.

---MikeTheRed 18:17, 22 May 2006 (PDT)

---

Thanks. No project names in research.dat -- but the order of projects on the Research screen matches the order in research.dat. Combined with the known values (mostly derived from the .cfg file included with xcomutil, I think), it wasn't that hard to figure out what everything was. The missing stuff was mostly research times for individual aliens, and I was able to run filecompare (fc.exe) before and after starting research projects on them to see which entries they corresponded to. Bytes 0x01 and 0x02 (zero-indexed) seem to match the values in astore.dat: 01 01 is a Sectoid Commander, 01 02 Sectoid Leader, etc.

I didn't try to figure out what all the bytes are, but if I remember correctly, some corresponded to whether the topic was available to study, whether the item/alien was in Stores... I seem to remember research time left (in hours) was stored in projects.dat.

The number of aliens you need to research to fill out the entire UFOPaedia is a little variable -- you can study aliens that give nothing, and early on, some will give two entries. I believe 11 corpses (180 ea.) + 9 live aliens (192 ea) (2 of which are Medics for live Sectopod and Cyberdisc) + 8 Navigators (192) + 8 Engineers (192) is a good approximation. This assumes none of the "9 live" are engineers or navigators, but also that there's a leader and commander in there: no efficiencies, but no redundancies either.

Your total time graph is useful (and it'd be nice to see it for w/ and w/o all the alien projects) -- the efficiencies part is esoteric, especially taken out to 10 labs, but to see 12 mos vs. 6 vs. 4 for 1 / 2 / 3 labs is quite informative.

Hmm, I should fix the note on the hovertanks a little, just got lazy on that point.

I'm still not sure how best to structure the Popular Sequences section.

--Ethereal Cereal 19:58, 22 May 2006 (PDT)

---

Ah, now I see... the old Notes 1 & 2 were confusing... all that they were saying is you also need Fighter Craft tech to build Hovertanks, period.

It sounds like you know your game files. To whatever extent you get the time, if the wiki doesn't have some of this stuff you've figured out, stick it in there. Even partial notes are better than none.

I had been thinking about optimal research sequences, although "popular" is also something to do. "Optimal" as in, what research route(s) gets you important goodies (psi, plasma, hyperwave) and good manufacturing a.s.a.p.

I'll see if I can't find that old total time thing and crank it out again.

---MikeTheRed 22:42, 22 May 2006 (PDT)

EC, I found my old total time spreadsheet. I can include those alien estimates (11x180 + 25x192, right)? I can also do variations, as you ask... maybe a "minimal three" path, the middle road you suggest, and "everything possible", and thus have three different total-time breakouts? What do you think. But if I do these, can you remind me / specify a little more what would be the differences inc. total aliens... I'm kinda rusty these days.

---MikeTheRed 11:50, 23 May 2006 (PDT)

I think there's only two sets of numbers that are particularly interesting: all useful technologies, and all UFOPaedia entries. Useful technologies is every research topic except Alien Entertainment, Food, Reproduction, Surgery, Examination Room, Corpses, and all the live aliens (except 3*192). All entries is everything except Reproduction plus 11*180 + 25*192 for aliens.

Run it for, say, 1-5 labs as well as 1 scientist and 10 scientists (the initial allotment).

--Ethereal Cereal 14:56, 23 May 2006 (PDT)

---

Now I see what you mean by Popular Sequences. Of course. Quite well done, thanks!

I can do both of the research total times you recommend. For a minimalist vet though, many things don't matter... superfluous aliens, plasma rifle/pistol & clips, etc. I guess I'll take a shot at defining minimalist and post the results.

Good idea to include 10 scientists; it's important in the early days - the most important part of the game. However, given the way columns can be easily packed into the wiki and that the spreadsheet's already constructed, there's no reason not to also paste columns with high numbers of scientists. It's not extra work for me. Hopefully nobody will ever make that many labs... the numbers make patently clear why they shouldn't.

I've put Civ4 down for the moment and am playing XCOM again. FWIW everything I wrote on the wiki was based on one game that I played last fall after having not played XCOM since what, 1995? (Not counting TFTD etc. after that.) As I neared the end of the game I started wondering about a number of things and hooked into the wiki and forums; the rest is history. I'm now on my second game in the last ten years. ;)

---MikeTheRed 22:01, 26 May 2006 (PDT)

---

Research Abuse

I disbelieve the "truncation" comments. But the following behavior would emulate it.

This game, I come into Jan 23, 1999 with the following from PROJECT.DAT:

Item	Man-Days	Scientists
Medikit	30h	4h
Heavy Laser	B7h	2Dh

So...throw 30h/48 scientists at the Medikit to get it on Jan 24. Reallocate research to 45 Heavy Laser, 4 Sectoid Navigator on the theory that Hyperwave decoders don't do me any good without money. [I'm putting cashflow ahead of armor this game.] And Heavy Laser now is....

Heavy Laser

8Ah

2Dh

45 scientist-days deducted, not one.

So (tentatively) the game [XCOM CE with XCOMUtil] does two passes, if not a loop. It first checks for research completion, then reallocates research, then applies the research. The question is what happens when the new projects include one whose scientist-hours are less than the number of scientists assigned...does research completion kick in immediately, forcing yet another assignment stage?

---Zaimoni 14:19, 19 July 2006 (CDT)

---

Sometimes quick projects assigned lots of scientists do complete immediately, followed by another assignment stage.

--Ethereal Cereal 14:48, 20 July 2006 (PDT)

---

Zaimoni, pardon my ignorance, but I'm not following well. Would you be so kind as to repeat what you said, but only use decimal numbers for ease of comprehension, and also say the exact times...

Did you re-assign a few minutes after midnight at the beginning of Jan 23, then viewed it a few minutes after the start of Jan 24? If you re-assigned 45 scientists at the beginning of Jan 23 and saw 45 sci-days deducted at the same time next day, I don't see a problem. But this is too simple; you wouldn't have a problem with that - did you save immediately, early Jan 23, and saw that? Or - did you do this re-assignment as of the moment you passed into Jan 23 (with X-COM having stopped the game there)?... if so, it suggests that ... anyway. If you could please supply a little more detail. :)

Also I don't know what the structure of RESEARCH.DAT is, and haven't tried to decipher it. The truncation effect detailed on the main page comes from actual observation of the game (samples) done by Zombie, not from reading RESEARCH.DAT (correct me if wrong, Z... I think more details are in old versions of the page or Discussion). Which brings into question what is RESEARCH.DAT showing... hours left, even for projects not yet started? (Does that mean they all get their ave/2 to ave*3/4 roll at the start of a game??)

If anybody can fill in more details for RESEARCH.DAT, that'd be great

---MikeTheRed 19:14, 21 July 2006 (PDT)

P.S. There's been a lot going on at work for me, but it's good to talk with folks here. I'm just curious about details on Research and how things work, Zaimoni! By the way we might get a good graphics app at work that will do 3D topo maps... but it will be weeks before it shows u, and it's not clear that we'll get it.

---

In decimal, the starting numbers (what I changed from on Jan 23) were:

Item	Man-Days	Scientists
Medikit	48	4
Heavy Laser	183	45

Ending numbers on Jan 24, six minutes after midnight, were:

Heavy Laser

138

45

Reassignment was indeed early on Jan. 23. The completion of the Medikit was on Jan 24/midnight, with counter changeover then.

This is probably an exploit, because 44 scientists were double-counted (first for completing the Medikit, then against the Heavy Laser). [This is XCOM CE w/XCOMUtil.]

PROJECT.DAT only shows scientist-days remaining for projects that have been started. I haven't bothered to decipher the exact layout; suffice it to say that everything is in the same order as in the research allocation screen. All of the time remaining entries come first (2 bytes each); all of the scientist allocations come next, in the same order, 1 byte each. I presume this is repeated for each base, but haven't checked that.

If you only research one project at a time, and fully reallocate at each project completion, truncation is emulated for all research projects except the first. And usually that pack of scientists coming in 72 hours after game start renders that exception moot.

EDIT: Oops, read PROJECT.DAT everywhere in my thread; I have fixed my comments. RESEARCH.DAT has been documented.

---Zaimoni 21:58, 21 July 2006 (CDT)

---

Thanks Zaimoni, that's making things clearer. I guess RESEARCH.DAT is a fixed look-up. Any details anyone knows for that or PROJECT.DAT, I hope they'll post to the wiki to help us all.

So the re-assignment happened on Jan. 23 right at the stroke of midnight, when a previous research assignment was up... And that's what you and Ethereal are going back and forth on - that X-COM may have programmed research completion in a poor way so that they might loop etc., eh? We've all seen examples of where they didn't quite get it right with other fine points, like personal lighting after Mind Control and extra reaction counts (see the second point # 3). These are clearly improper sequencing of events (but have little impact, or they would have been fixed).

If PROJECT shows the time remaining, it would probably make an easy way to sample how research works, by reading savegames. Like I said though, AFAIK Zombie was only playing "in game", neither editing nor viewing any savegame files when he made conclusions about truncation. It could be influenced by this new wrinkle you bring up... It's entirely possible he did them one at a time, and could have missed something like this.

I have recently realized I made a vacuous statement about how often names are duplicated in Soldier_Name_Stats. What I actually did was I started with a particular savegame, ordered batches of 100 soldiers, and saved these 100 to a dataset immediately on arrival. I repeatedly used the one savegame, and never ran one batch into the other. Thus, although it was easy to collect the data into a database, I ran rough over the fact that the game might not have duplicated names if I had not started with the exact same savegame, recruited exactly 100, etc. In other words, it might not have duplicated them if I, instead, compared one batch of 100 against a second batch recruited after those first 100 arrived.

Testing can be tricky, eh? I'm sure Zombie can say whether or not this issue might affect his results. Somebody page our favorite tester, eh?

---MikeTheRed 21:00, 21 July 2006 (PDT)

[so-called vacuous statement] I'm not too concerned about that.

The savegames do not save the random number generator state. Otherwise, you'd get exactly the same results from the same reload, which would be very abusable.

The usual way of initializing a C library psuedorandom number generator in DOS and Windows, is from something derived from the system time. [*NIX probably would use dev/random or similar.] As long as your batches were started at different times, the correlation from starting "close" in time should be nothing compared to internal weaknesses in the programming library's algorithm.

Said weaknesses were exposed in your experience testing.

---Zaimoni 2:34, 22 July 2006 (CDT)

---

MTR, Zaimoni: Indeed, I only tested one research project at a time. Why? To eliminate variables. (And for those of you who know me, eliminating variables is my middle name while testing). Looks like multiple projects is a variable (or perhaps exploit) after all. And if this is the case, then yes, it will affect my results as I only took one project into consideration at a time. --Zombie 00:14, 23 July 2006 (PDT)

---

Zombie,

I did validate the scientist-day numbers you measured, for projects that were not started at the midnight another project was completed. The complicating factor appears to be starting a research project exactly at midnight, which is very hard to do without completing a project. Automatic application of effort doesn't happen if the project is started at another time. [Hmm...what if a project is started exactly at midnight without completing another project?]

---Zaimoni 23:57, 23 July 2006 (CDT)

Scientist Double-Counting

Finally got in some more experimentation...turns out there is only one iteration of the research loop at midnight, in "increasing order".

That is: if a topic completes, reassigning scientists to a topic earlier in the enumeration does not cause double-counting. (If you reallocate scientists from ahead of the completed topic to behind the completed topic, you've just shafted yourself -- you will get no research from those scientists.) Only reassigning scientists to later in the enumeration from earlier than, or the completed topic itself, allows double-counting of scientists.

Of course, there was an ego trip from completing seven topics at midnight....

---Zaimoni 14:30, 13 Sept 2006 (CDT)

Just wondering about the way the double counting works - is pretty much everyone who plays the game using this bug some of the time anyway? Basically any time you go into research after a completion (which I assume most everyone would do anyway), if you reassign those scientists to a new (or existing) project, if its lower in the enumeration than the project time update has got to so far it would get a bonus round added into it during that same update, which would explain people without even knowing about this feature and deliberately running multiple projects to abuse it also getting instant wins for short projects that are reasonably far down the research list. As far as I can see the way it looks intended to work, instant project completion must mean this bug has been triggered, but also it was a project with a total time left <= your number of assigned scientists.

--Sfnhltb 00:21, 27 February 2007 (PST)

Yes, everyone who behaves normally will sometimes use this bug. Just to be clear, it's if the reassigned project is higher in the order than the completed project. This happens to be true for all research prerequisites. So, for instance, if you start researching Laser Pistols when completing Laser Weapons, then you're using this bug.

And yes, instant project completion means exactly what you stated.

---Zaimoni 09:59, 27 Feb 2007 (CDT)

---

By higher in the order, I guess you mean higher in terms of the offset in research files and such, which equates to lower down the on screen research list?

--Sfnhltb 10:50, 27 February 2007 (PST)

Note of course when you mention research prerequisites will always get this bug, this is true for technology prereq's, but as all the alien interrogation techs are at the bottom of the list, whenever you get new techs from them, you wont get the double counting if you immediately then start on the topic unlocked (Alien Origins, Hyperwave, Martian Solution, etc). The only things you can reassign to after an interrogation is another interrogation that happens to be lower down the table.

--Sfnhltb 18:31, 27 February 2007 (PST)

This sounds interesting, but I'm not sure I'm following. Zai, do you have to have more than one project running? I always work only on one project at a time. - MikeTheRed 15:18, 27 February 2007 (PST)

No, you dont need to be doing multiple projects, basically what is happening is that at midnight it (among other things) updates all research progress. Ignoring multiple science bases (which I am not sure how it handles) it goes through each of the projects in order, starting with Laser Weapons if you have any scientists working on that project, it takes that many off the number of days of Laser Weapons research you have remaining. If that is now zero or below, it completes the project and shows you the reassignment page. Once you have finished reassigning, starting new projects, or whatever and press okay, it goes back to updating research status on the next project in the order, which would be the Motion Detector. So if you moved the scientists who just completed Laser Weapons onto the Motion Detector project, they will now also help out on the project, whether it has only just been started or was already in progress (with or without other scientists working on it).

Conversely if you completed Motion Detectors, and in the reassignment immediately after that you moved them onto Laser Weapons, then they wouldnt get this bonus scientific production, because by the time it does Motion Detectors (research item 2) it has already updated Laser Weapons (research item 1). They are potentially 94 or something other projects (some cant be researched without hacking, if then, though) that you can reassign the scientists completing Motion Detectors though, such as Medikits (research item 3) or Reaper Terrorist (research item 96). The Reaper, being last, cant ever cause any double counting [edit: when you complete it and reassign after, that is], everything else can with at least one other project (the Reaper), although its unlikely you can do absolutely everything in this order (or would want to for some projects).

Note that once the clock has passed midnight and you have gained the bonus research, there is nothing stopping you then reassigning all the scientists to something else (in some cases you might have nothing you particularly want right now that was beneath the item you just researched, so you might just take the double counting and score it against some low priority project, so that when you get around to it, it will take one less day to research (well not always, if you hire more scientists before finally doing the project you double counted against it may or may not save a day).

--Sfnhltb 17:41, 27 February 2007 (PST)

Ah cool, now I see. Thanks for the good explanation - why not put it on the Research page? It, too, is useful stuff to know. (Somehow I missed or forgot about these entries last summer, Zai.) One final question, just to be clear - when these scientists are re-assigned to a higher-sequence project, do they take ALL their Man Days from the project they just finished, or only the MDs from the day that just ended? Thanks! - MikeTheRed 18:07, 27 February 2007 (PST)

Well that dont 'take' the Man Days, as the project still completes, and you only get one extra man day per scientist out of it. Well thats per completion reassignment, in theory the same scientist could be reassigned 20+ times in one night if you happened to have enough projects lined up on the verge of completion, as long as you do the reassignments in the right order. Essentially if you assign to a project that is checked later, it treats it as if he had spent the entire day working on that project as well, whereas if you assign him to a project that has already been checked for this day, he wont do any more work until the next days research update.

If you dont want to use this bug, all you have to do is never reassign when you complete projects, and just go in any time in the next day and reassign them at that point - as scientists can go on a round the world trip from immediately after midnight, spend 22hrs in transit to another base and back, sit in the coffee room for 1hr59, then get assigned a new piece of work, and the scientist will still manage to put in a full days work. I think we might be overpaying them at 30k a month, considering that.

--Sfnhltb 18:27, 27 February 2007 (PST)

MikeTheRed: I normally have more than one project running, when inspecting PROJECT.DAT. It changes the game dramatically to routinely have complete information on the status of research. Sfnhltb's restatement is correct.

Sfnhltb: Research is already known not to be cumulative across bases, so I haven't tested the research order between bases; it's simplest just to use Base #1 for all research. I believe the bases are "processed in order", as manufacturing is updated in increasing order.

--Zaimoni 9:49, 28 February 2007 (CST)

Yes, I agree, if you can instantly interrogate a save to see that there are 84 MDs remaining on a project, so you reassign 34 for the first night (and have the other 16 do something else), then 50 the next day so they can complete it an then all double up on something else for each completion it can quite noticeably boost your progress (hardly surprisingly).

I might set up a test case to see what happens with the multiple science bases in a few different scenarios at some point, but I guess it isnt much of a priority as its rare to have more than one team as it just adds hassle and potential wasteage, usually my first base is the science base and I never build any in the new bases at all.

--Sfnhltb 09:25, 28 February 2007 (PST)

Thanks for the clarification Sf. I'm just wondering if all this work (progress bar, double-counting) can be used by the savvy player who isn't actually looking under the hood. Aside from the idea to work up through the research records, is there anything else that can be done with it? A couple more thoughts/questions:

• The number of "cheesy free" MDs will always be equal to the number of scientists that just completed the project, right? BTW, I think somebody called them "lost" if you went to a lower-number project. But to me it sounds more like they're treated normally, i.e. not abused, instead of lost. I didn't miss anything, did I?
• Question: Let's say you just completed Project #20. Now you start #21, and see Unknown. Can you cancel it and try it again, to see if you don't get Unknown? Or cancel it, try #22, then try #21 again? Or is the next research roll fixed, no matter what you do that particular midnight? (I haven't been playing lately... does Cancel drop you totally out of that night's Research?)

MikeTheRed

Sure, some of it. Progress bar - only the unknown bit helps here, if its unknown it is between 2/3rds of the average to 150% to go, if you have a rating then it is down to 2/3rds or less to go. So you have that to give you a vague idea of how long is left, assuming you know how big the project is (eg Heavy Plasma is 800, 2/3rds of that is 533, so if you have 100 scientists on it you can tell on the day it switches to average instead of unknown that you have 5-6 days left before completion (10-11 for 50, which would read poor not average). This could help you plan for saving up for production/facility build cash/requirements and the like.

The actual rating once seen is fixed by project average/scientists, one of which you can look up on this site, the other you know, so it isnt particularly useful except as a reminder of the scale of the project you are working on without having to go to an external reference.

The free MDs are created by the fact you reassign the scientists that were working on project A to Project B before B is checked, so assuming you actually put them all on the new project, then they all double count. If you only assign half to a project, and leave the other half unassigned at that point, only half will be double counted. And yes they arent "lost", they are just single counted (well, in fact if you dont know the time remaining on a project, on average half of them are never counted against any project, as they are overrun on the initial project).

Im not sure I would say they are treated normally though - someone who has no knowledge of this bug probably uses it 60-80% of the time anyway, because the vast amount of people will reassign immediately when the window comes up, and more often than not, the next project is going to be lower in the list (partly because of pre-req's).

You cant cancel a project in the interface (you can hack it out of the save file of course, by zeroing the time remaining), so the only way you could do this is have science labs in multiple bases so each one could try a project, and if one gets a non Unknown, then that one does that particular project, I guess you could try 8 times against each project if you build labs in every base with 1 scientist each (to do the poor/unknown check), and then fly the bulk of your scientists over to whichever lab turns it in 'cheap' mode (1 in 6 chance, with 8 attempts, will get most projects in cheap mode). Flying them takes at most 16 hours i think, so you wont lose any research time from travel.

If you cancel out of reassignment (either on the first screen, or by exiting the research screen before moving some/all your scientists then the count as unassigned and wont count towards any further projects that night, or in fact ever, until you go in and reassign them. If you somehow happened to complete another project as well that midnight you would probably get another chance to reassign (and potentially double count) with them again, but I havent tested completing multiple projects in a single night yet, so cant say for sure.

--Sfnhltb 17:35, 28 February 2007 (PST)

Progress Indicator work

Added some detail about the way Progress works on the current research screen as no one seemed to have nailed it down before.

Removed the following discussion from the above section as it was redundant now:

However, these other ratings have not been quantified. Furthermore, Unknown seems to be the most common initial status. The upshot is that you get a little lucky sometimes.

Further testing suggests that the "Poor" to "Excellent" ratings might be considered a "Progress Bar" relative to the time that the project will complete. It is possible for a project to complete at midnight on a day where it was Excellent, but you stole away just enough engineers to make progress be Good instead of Excellent - but this is for very short manhour projects. For long duration projects relative to your # of scientists, it is more likely that you will have to go to Excellent to complete. This might or might not be able to be turned into something usable by players; I think there are two variables at work here. More observations requested - MTR Hey NKF, I like the "small" font. Remind me how to indent a paragraph like this relative to the bullets around it? BTW I think I misspoke a week ago - it's you that did the melee testing, not Zombie, right? Apologies for my poor memory... I was working with Z so much back then.

For MikeTheRed, to indent a paragraph start it with a colon, or more than one for extra layers of indenting (probably found this out a long time ago no doubt, but others might not know).

Note I did all the experimenting on getting the breakpoints by changing the values with a hex editor, so there is a chance that when these are changed naturally by the program some other values (maybe in RESEARCH.DAT) change as well, so you might see variation from what I observed doing it this way, but I dont imagine this is the case (and it all fits in with my experience playing the game, i.e. things like you only get a higher ranking with more scientists assigned, how long they work on the project getting it closer to completion is irrelevant.

--Sfnhltb 14:24, 26 February 2007 (PST)

---

Great stuff, S. I always wondered how that worked. I stuck in an example for that one place - I had to think about it a minute, so I figured others might, too. Hope you don't mind.

Everything you've posted sound consistent with my experience. It's good to know how it works - and get a tip on how to know if you just got lucky, hehe - MikeTheRed 16:16, 26 February 2007 (PST)

Yeah, I was thinking as I was previewing it to check it all that I probably wasnt being as clear as I could be in places, but then thats the beauty of Wiki, as long as a few people can understand what you have written they can add a different take, or adjust it, and this can be repeated until it becomes clear to as many people as possible. In theory.

I guess the other thing that could be added is working out how long (as a fraction/percentage) a project will normally be in Unknown, given starting from a random roll (and assuming the same amount of engineers on it the entire time of course).

Lets see, 16.66*% of projects will be 0% of the time, then it ramps up to a 150% project needing to take off 83.33*% of the average project time before it gets a status, or 55.5*% of the total duration of the project. It isnt a linear function though, as for example at 100% a project spends a third of its time as Unknown. I imagine theres probably a way of working out the average, not sure it would be too useful anyway though.

--Sfnhltb 16:48, 26 February 2007 (PST)

Page split

I think this page needs to be split, probably into at least three sections. May reread through it soon and try and come up with a reasonable way of splitting. I am thinking initially that the list of research times should go to a seperate page first, including the comments about how to use them at least, and maybe including some of the analysis as well. Another page with the research trees could work as well I think, with the idea here being to leave this page as an overall Research page that then drills down to these subtopics, maybe with a brief paragraph left to summarise key points about the subpage.

The page might just need a reorganization, not a split. We could do a better job of presenting the need-to-know info at the top, then break it out into more granular detail section-by-section. My initial suggestion would be to group "Popular sequences" and the research trees together. Maybe even overlay the total-time figures onto the existing trees... although that might make things harder to read, unless it was done just right. The "Research time" charts should probably be grouped in with these as well.--Ethereal Cereal 02:34, 27 February 2007 (PST)

Well one thing to consider - we have an area on the first page with Data Tables, so an entry there with Research Times would seem to make sense, and then those two tables could be moved out - and of course linked numerous times from within this article.

Does this sound reasonable?

Obviously the main issue in any wiki is if you split the pages without clear delineation between what the content of each will be, and ensure they heavily interlink, you rapidly get much duplication between the two pages (well rapidly in comparision to the amount of edits anyway) as people see something 'missing'.

The problem I see is that the idea of hypertext/wikilinks, is that you provide core information about the topic/subject on the top page, and then for those interested more detail can be found on subpages that are linked to, which you can make as many as you like of. This allows you to cater for people that want to know everything; they can just follow every link until they have exhausted all the knowledge/information available, but also the casual read that wants to know the basics then doesnt have to scroll through 27 pages of screens and try to pick out the key information from the more detailed stuff.

--Sfnhltb 10:16, 28 February 2007 (PST)

I'm glad to see you two working on this. I think it could use some re-organization, but agree as I think you two do, that it needs to be done carefully to avoid duplication. Fortunately it does seem a bit more granular than e.g. the very long Explosions page. But the Explosions page has less granularity/more overlap. (It could definitely use some condensing though; it suffers a lot from "learning as we went".)

I do think there are some good pointers up above that could use being on the page, such as scientist double-counting.

Anyway. Somebody's mission, should they decide to accept it, is to artfully re-organize the Research page(s), LOL - MikeTheRed 15:58, 28 February 2007 (PST)

I've been pondering exactly how to approach this for a few days. I'm okay with moving out any "under the hood"-type info, only that stuff is pretty small -- parts of Total Research Time and Research Progress. Everything else in the article strikes me as being immediately practical, and would be less so if it was cut into two pages just on the basis of length. I don't like the idea of putting the Research Time tables on a separate page, for instance -- on some level it's the most practical part of the entire page.

I'm starting to see that the page could be split into reference and analysis/tips sections. But I still think it'd be more practical if both were on the same page. I agree with the wiki "subarticle" style but not with a "coarticle" approach, which I fear is what we're facing here. Raw data begs for a "what do I do with this info?" and tips need to start with the data as a foundation.--Ethereal Cereal 03:11, 1 March 2007 (PST)

I guess my main concern is casual readers are not going to read this due to length, firstly a full page and more of contents is going to put many off, and then because the key/basic information of the page is split through the rest of the other 27 or so screen heights, its going to put off all but the most dedicated that are willing to pretty much read through it all to get all the salient points out.

What I would suggest is that all the top level sections that already exist need to stay, so we would have a research times section still, but it would describe the basic elements - that they are fixed once you start a project, they vary randomly based on an average, and so on. This section would also clearly include a link to the full lists of times as a seperate page, moving the detail away unless it is required, then it is only a click away at the point it is likely to be foremost in the readers mind, if they are inclined to find out that sort of thing. Repeat this for the popular sequences - have a description of how certain technologies group together, maybe list the main groups, mention how some of these group overlap, etc., then point to an article describing all the groupings in detail that we have now. Take away the section I added on research progress as well as another article, and add a one or two sentance description and a link to it (in the research times subtopic on this page I would think is most natural place). Move the sections on total research times and scientist efficiency and the like to the research times page as well I would say, but that might need some consideration, for me its too low level/detailed for a top level article of this type (and length), but I am not sure the best place to put it, leave it to the end maybe.

Split off sequences as well, and then add a section back with a description and a link. Seperate out the researching aliens details, and put back a section that is to the point - describes the process in general, picks up the key points (navigator, psionic capable) and leaves the rest to the more specific article. Combine The Minimum Three and Capturing Live Aliens into that one article maybe, as that is all interrelated. Maybe not though, would have to reread in detail to be sure.

The main thing would be to be left with a much more tightly focused main article - to highlight all the basics, and the key information about research, and then link in to all the other stuff that is more specific.

For me a resource like this is likely to attract two types of player - one group will just want to play, but want to pick up some tips where they got stuck maybe, or get an introduction and starting point in what is a fairly deep strategy game, the other group (most of the people that are updating it, I would guess), want to tear the game to pieces and work out every detail to the nth degree. Because of this its best if you can create a 'seam' between these two groups of information that gets accumulated. For short articles/topics, the introduction and maybe the first couple of paragraphs should explain all that is needed, and then it can go into the more detailed items in the rest of the article. For more complex topics, like research, there is certainly a full article there just to cover all the basics properly, so then the detail needs to be hived off to avoid diluting the key points.

--Sfnhltb 04:05, 1 March 2007 (PST)

I agree with you there on our two main target audiences. Often I feel some of what we put in here may be a tad too much for your casual player. Unfortunately, for me at least, it's hard to write from a laymans' perspective - I mean it's difficult to remember what it was like to be new at this game considering all the knowledge that has been picked up over time. I suppose some sort of executive summary at the very beginning of most of the more detailed sections would certainly help clarify the topics.

By the way folks, for the discussion pages, could I ask if everyone please make judicious use of indenting paragraphs (multi-indent if you have to) and white space. It's hard to tell where some of the intertwining discussion threads begin and end at times.- NKF

Executive summary... now that sounds like a good idea. It's a moderately big rewrite we're facing here, but I want to tackle it (and steal Sf's thunder, heh). In the near future, I intend to attempt it, using the following structure:

• Executive summary
• All the really practical advice
• The gory details in full

I think this is what Sf's been driving at. I agree with a resectioning like this, just not, like I've been saying, two or more articles. Properly restructured, I think a single page can best serve both newbies and hardcore players.--Ethereal Cereal 22:51, 1 March 2007 (PST)

I lean a lot toward what you're saying, Eth. I quite dislike it if there's overlap and duplication. Thanks for accepting Mission Impossible, smile. But you've always had a gift for wording and re-organization. I love what you did to the Main Page, even though I doubted it until you did it.

FWIW I just happened across a Civilization 4 wiki (which I've resurrected and am playing now). Take a look at the interestingly concise way they organized it. However - and I'm just getting back into Civ 4 - I think they haven't really dived deep, like we have.

NKF, re: indentation, extra line spacing, etc.: I am often not sure when a new topic is ending or beginning, in Discussion. And it seems a touch lame to always go one more indent to the right. In theory these many Talk pages with 100 lines should be far off to the right, hehe. Come back? - MikeTheRed 23:04, 1 March 2007 (PST)

If you look at "the way it's done on Wikipedia" you'll see it works reasonably well. If things get way too indented, somebody does a "resetting indent" and the conversation continues. The biggest thing is that they use headers for each topic (and people reorganize ongoing discussions if they start getting too convoluted). The style that was adopted here early on -- a line between each post, no indentation, no headers -- has its shortcomings.--Ethereal Cereal 01:29, 2 March 2007 (PST)

Cool, let's give it a go. BTW I'm still sitting on a ton of Firing Accuracy data, using BombBloke's automator - I won't be able to get to it any time soon, but it's several weeks worth of data. If anybody wants it, just say the word. - MikeTheRed 09:00, 2 March 2007 (PST)

Restructuring attempt one

Feedback plz.--Ethereal Cereal 00:02, 3 March 2007 (PST)

Its already a lot better like this. The one thing I would say, even if we want to keep all of the detail of the tech trees and such, can we get them to not appear in the toc as it is excessively long imo.

Another possibility would be to combine the information in the tech trees and the popular sequences together, so the trees had the times to research from scratch all the way down, and put the most important items as highlighted in the popular sequences in bold (or with a note, as bold links might be a bit odd looking) and add the comments from popular sequences about those important items under the trees.

--Sfnhltb 14:02, 3 March 2007 (PST)

This is what I mean, roughly.

--Sfnhltb 14:23, 3 March 2007 (PST)

Yeah, I suggested that at first myself. But ultimately I think I prefer the semi-redundant solution of separate "popular sequences" and "complete trees". Otherwise you have to pack too much info into the diagrams, making them do two jobs more poorly than the separated lists do.

I replaced the L3 headers with "big" font to remove them from the TOC. Looks perfect on Firefox, acceptable on IE. There are certain aspects of HTML (like tables) that truly, truly suck.

What else needs doing?--Ethereal Cereal 17:21, 3 March 2007 (PST)

Much better, how about combining "Capturing Live Aliens" and "The Minimum Three" into one topic, its almost in reverse order at the moment. I guess the general flow I would think best would be: Why you capture Live Aliens, how to do it, the key three you need, and then details on what the rest can add to the UFOpedia --Sfnhltb 17:33, 3 March 2007 (PST)

Regarding the last point, I mean the general types of information they can add, and then maybe also include a link to the point to later in the page where the exact list is details for each type of info. --Sfnhltb 17:35, 3 March 2007 (PST)

I started in on that (and tweaked a lot of other parts too), but I found that the "why" was best answered by the info in The Minimum Three, so I left that in front, and changed "Capturing live aliens" to "How to capture live aliens". I also added a sentence to the end of The Minimum Three noting the other benefits of alien research with a pointer to Researching Aliens later in the page. Anything left?--Ethereal Cereal 22:50, 3 March 2007 (PST)

I wonder if it will be better if we can have a section on researching aliens, and instead of Minimum Three, call it something more obvious like Priority Targets or something, it can still go first if desired. The problem if this is done is I think a number of places sublink to that particular section name so the are going to break and theres no easy way of locating them (except by memory or brute force of checking every link to this page). For most articles that wouldnt be so important, but theres so much content here that being left at the top of the article is going to be problematic for people new to the subject matter. --Sfnhltb 18:51, 7 March 2007 (PST)

Go ahead and change it according to your tastes, but give me a chance to look at it before you change other pages to match it.--Ethereal Cereal 01:57, 8 March 2007 (PST)

Tables

I converted the research lists to tables, cant really do the same to the popular project sequences as easily because of the alternates at the bottom of some dont fit into a neat table structure so would have to mess around manually with it most likely, and not sure it will look much better with merged cells like that anyway. --Sfnhltb 18:54, 7 March 2007 (PST)

I can't help but feel that the blockquote tables were more readable than these html ones...--Ethereal Cereal 02:11, 8 March 2007 (PST)

I never particularly liked wiki tables. Look at all the horizontal eye movement needed relative to the wide columns. Any fix for that? Hey the page looks fine - thanks for all the work! - MikeTheRed 21:31, 9 March 2007 (PST)

---

Simple. Don't define the table width. This fits the data to the cell with no extra space. I normally don't do this as each table in a page is a different size. Depending on the amount of data, I usually stick to 25, 50, 75 and 100% of the relative width of the table to the page. If the table falls nearly on one of these breakpoints, I round up to the next size. This way there is still some space between columns and the table isn't cramped. Of course, width sometimes depends on the overall look of the table. If it looks bad, fiddle with the size until it does. Also, centering a table is kind of unnecessary. Either justify it left or right (left is better) and leave it be. (Another reason why I don't like to center things is because the center tag is deprecated in HTML and not even supported in strict XHTML. Not that this matters much in wiki coding but it is something to consider). My 2 cents. --Zombie 22:27, 9 March 2007 (PST)

---

Sounds like it could use a little padding, and otherwise let itself be as small as it might, relative to the padding. Like, a couple of spaces on each side of the widest datum in the column. But then, one might want to shrink the column heading (to wrap it a few lines) if the data itself is stricly, e.g., "1" "2" or "3", but the column header is wide. How can this be done?

Like I say, I never liked wiki tables. Happy to have someone show me how to do this. And then after that, show me a site that lets me dump MS Office data into it, and convert it to be a nice tight table. After all these years, ASCII skills still seem to be useful, hehe - MikeTheRed 22:55, 9 March 2007 (PST)

---

Part one: If the data isn't as wide as the the heading you could do a few things. 1: Abbreviate the table header which is too wide. 2: If the table header cell contains multiple words, lower the width of the column/table until it wraps to a second line. 3: Don't define the width of the table and let the column be as small as possible.

Part two: There is no way to export data (from an Excel spreadsheet, say) into a nice html format. Well, that's not entirely true. You can export the data into some form where you can replace parts of the format in notepad (say a comma between cells into /td td and a line break as /td /tr. Or as I read one time, replace each line break as /td /tr tr td> then split off the opening tr and td tags and dump them on the next line to start off a new row). Sound complex? It is. I find it much easier to just code it manually. Importing web tables into Excel is actually really easy so that's why I prefer creating a table in html first and then importing it to Excel. ;)--Zombie 23:17, 9 March 2007 (PST)

---

Thanks, Z. There are websites out there which will convert various things. But as you've said, it's easy to go from html to Excel, but what I want here is, the other way. One thing to point out is that I have Office v. 2k; perhaps 2k3 or later versions will be better; xml isn't even an export format in v. 2k. But I know it is in 2k3 (although I have that at work, not home). So maybe things will get better, if I ugrade to 2k3 or later.

In the meantime - as I think we've agreed if you've seen tight tables - wiki tables can't be made tight, but are a decent quick and dirty way to present info. Unless someone with more wiki tips knows more? - MikeTheRed 23:29, 9 March 2007 (PST)

---

Technically, these aren't wiki tables but plain old HTML tables. The wiki table markup {|, ||, |} etc. just converts stuff into <table> <tr> <td> </td> </tr> </table> etc. And on this page, Sf has mostly used non-wiki HTML table code anyway.

HTML tables fucking suck. You've got to stand on your head and know a million different little tricks to get them to look right. One of my biggest annoyances is you can't specify properties at the column level (like right/left/decimal alignment, etc.) unless, again, you stand on your head.

Regarding spacing, two tricks I've learned are adding style="padding-right: 50px;" to the TD in order to add a little whitespace if it looks too compressed; another trick is to add <br> to long entries (like headings) to narrow them without having to manually set widths.--Ethereal Cereal 00:31, 10 March 2007 (PST)

---

I use Excel to make these tables, not via any export function, but just by creating a function that concatenates the various tags needed on each row around the content, this way its easy to play with until you get something that looks okay without having to manually make a change to every line for each change you think might help. Problem is with most of it is that it can look fine on one screen width/font size, and turn into a complete mess or have large gaps on others, but thats the limitations of html unless you take the time to mess around with it for some time and test on various systems/setups/browsers. --Sfnhltb 11:25, 10 March 2007 (PST)

---

I tend to use python scripts myself. Just copy-paste the data into a normal text file. Usually I grab it from the wiki source because there are nice separators to split. Other times if I don't have access to a wiki-source I put it in Excel and save as CSV, then parse with python (I guess I could have python parse the HTML, but too much work).

From there I just arrays and whatnot, play with the data as I need it and then write a loop and concatenate the needed mark-up. (Pretty much what Sfn does, but I like the power of a python interpreter). Also makes it easy to add in style="..." in. Also it's how I split long lists into multiple columns by using more code. If all else fails you can make some regex's to do whatever needs to be done, but I haven't needed that yet.
--Pi Masta 12:31, 10 March 2007 (PST)

Well I went ahead and converted Sfn's table to the Wiki markup. I think it looks better and easier to read in the wiki-source than tons of ,,, tags floating around. If you don't like it you can revert, but ITT it's better with the columns lining up in the wiki-source. One thing I don't like about this mark up is you have to end |- tags with a new line and then put in the cell content, thus there's always an extra line between rows making the table long in the source.

BTW The meta wiki has a good page on table formatting http://meta.wikimedia.org/wiki/Help:Table
--Pi Masta 15:48, 10 March 2007 (PST)

---

Sure, I read raw html so often I dont think twice about it, but it works either way for me. I also agree on the extra line required offsetting some of the advantage of the wiki variant. --Sfnhltb 16:55, 10 March 2007 (PST)