There have been a couple of recent (April-May 2013) discoveries that affect the need for or use of some of XCOM Modding tools. They are summarized here for reference.
Phoning Home
When using mods, it is necessary to prevent each game from connecting to it's update servers behind your back. However, the DNS entry is clearly for a vendor specific address. This appears to be a vendor verifier (i.e. "Firaxis Verifier", in this case) independent of those patches distributed via Steam. They may not be 'patches' per se, but merely replacements of certain files to ensure consistency. This is informally called 'phoning home' and initially was believed to be intended to be a Steam mechanism to prevent cheating in Multi-Player games. However, it has now been determined to affect single-player games as well.
PatcherGUI (as of v5.2) includes options to "Enable INI loading" and "Disable phoning home" for you. This is the easiest and recommended method.
If you use PatcherGUI, you can skip the rest of this sub-section. It's intended for those who have to deal with this manually.
Those interested in the mechanism involved in this 'phone home' process are referred to the Class: IniLocPatcher - XCOM:EU 2012 article.
This edit to the hosts file is not affected by Steam delivered patches or re-installing Steam and does not affect your ability to get updated when you go back to 'Online' mode for ANY reason (such as connecting to the Steam Store). Note this should be used in conjunction with disabling 'auto-updates' for your XCOM game in the Steam Library. Be aware, however, that since November of 2013 XCOM patches seem to disregard this Steam 'auto-updates' setting and have been known to apply regardless. Having a backup copy of your complete modded XCOM game tree current and available elsewhere is recommended as well.
At present, the EU 'phone home' process is known to overwrite Armors, Characters, and Weapons arrays of the EXE internal resource cache version of the DefaultGameCore.INI file, and the ...\My Documents\My Games\XCOM - Enemy Unknown\ file XComGameCore.INI file which is a merged copy from the DGC.INI and DLC content.
- My Games\XCOM - Enemy Unknown\XComGame\Logs\EMS\
- XComDLC.ini (XComGame.XComDLCManager)
- XComGame.int (XComGame.XComDLCManager)
- XComGameCore.ini (overwriting Armors, Characters and Weapons arrays)
- XComMPGame.ini (overwriting a lot of MP related arrays)
These may be in My Games (if you have DLCs installed) or the <Steam install path> equivalent path, or both. If you are having problems with mods that alter one of these files not seeming to 'stick', check that your hosts file has the correct servers disabled according to the Launch.log as described below.
To determine where your game attempts to communicate, you want to examine your \My Games\<Steam Game folder>\Logs\Launch.log file. You are looking for lines similar to these:
The entries marked like so are:
- The Server IP address: 65.118.245.165
- The Server DNS Name: prod.xcom.firaxis.com
These are what you will place into the hosts file to block attempts to connect.
- Start Menu > Right click on 'Notepad' (or any text editor), select "Run as Administrator"
- File > Open > %systemroot%\system32\drivers\etc\hosts. The hosts file has no '.' suffix, so make sure you don't add .txt or anything else. Also, %systemroot% on most systems is C:\Windows, but the exact folder name may vary by OS version. The value of the global environmental variable %systemroot% on your system can be determined by entering 'set' on the Windows command line, or as it's shortcut is called, the Command Prompt. But the variable %systemroot% (including the percent "%" signs) can safely be used without knowing it's actual value.
- Add these three lines to the hosts file:
- (The first line is a comment. The next two lines do the actual work, but are specific to XCOM:Enemy Unknown 2012. Other games may have their own DNS names [i.e. prod.xcom.firaxis.com] and IP addresses [i.e. 65.118.245.165]. The '127.0.0.1' IP address is a local loopback address the prevents attempts to connect to either of the other addresses from ever leaving your computer.)
# Prevent XCOM:EU from Phoning Home
127.0.0.1 prod.xcom.firaxis.com
127.0.0.1 65.118.245.165
NOTE: The "Enemy Within" (EW) expansion to XCOM:EU 2012 uses it's own DNS and IP address entries:
# Prevent XCOM:EW from Phoning Home
127.0.0.1 prod.xcom-ew.firaxis.com
127.0.0.1 65.118.245.139
However, early reports (13 Nov 2013) that disabling these addresses in hosts will prevent EW from launching have been attributed to other factors, such as failing to also disable 'auto-updates'. Save game synching with Cloud Storage still functions correctly, even with these addresses disabled.
The XCOM-EW expansion is proof that other DNS and IP addresses can be utilized by the same or different games. Care should be taken to check the Launch.log file as outlined above for changes with each patch or major release.
Disabling Hash checks
Enabling INI loading
PatcherGUI (as of v5.2) includes options to "Enable INI loading" and "Disable phoning home" for you. This is the easiest and recommended method. See also the INI Loading Test entry.
The following is provided for background and those interested in the manual method.
A recent (24 May 2013) discovery makes it possible to force the game to load the DGC.INI config file from the "<Steam install path>\XComGame\Config" folder, thereby eliminating the need to use ModPatcher or make DGC.INI changes to the EXE at all, and avoiding the size limitation on the embedded file. (See the entry note on ModPatcher here about size limitations.)
Bear in mind: Loose INI files get read, and thus override mod changes to the internal 'resource cache', after the game EXE starts.
Choose your mods intelligently and with understanding. Not all will work together.
These hex offsets are for EXE version 1.0.0.28586; patch 4, Changelist 356266, unless otherwise noted.
- In xcomgame.exe: starting at location 0x157D93A.
- For Mac OS XCOM Elite Edition: starting at location 0x1D86A70.
- For Mac OSX Steam version, patch 3 (Jan 8, 2013): starting at location 0x1E7E5E0.
- For Mac App Store version 1.0: around offset 0x1BD58D0. Search for "72 00 00 00 65 00 00 00 2E 00 00 00 69 00 00 00 6E 00 00 00 69 00 00 00" (which is "re.ini" in UTF32 Little-Endian).
NOTE that these strings are also found as part of disabled debug code, at offset range: 0x156BA82-156BF38.
- For Mac App Store version 1.0.1: starting at 0x1D86B50. If searching in a hex editor, look for "44 00 00 00 65 00 00 00 66 00 00 00 61 00 00 00 75 00 00 00 6C 00 00 00 74 00 00 00 47 00 00 00 61 00 00 00 6D 00 00 00 65 00 00 00 43 00 00 00 6F 00 00 00 72 00 00 00 65 00 00 00 2E 00 00 00 69 00 00 00 6E 00 00 00 69 00 00 00". This is "DefaultGameCore.ini" in UTF32LE.
If you edited version App Store version 1.0, the App Store will not update the game and you will need to delete it and re-download. After editing, the app will need to be code-signed, again (as usual...).
Take care with the location of the strings you disable. As always, these locations may change after official game patches are released.
There are the three unicode text strings:
- \.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.G.a.m.e.C.o.r.e...i.n.i...........\.....\.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.M.P.G.a.m.e...i.n.i.......\.....\.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.L.o.a.d.o.u.t.s...i.n.i.
- It appears that these three strings are checked to see which files to NOT load from the Config directory.
- Change the first string to something else:
- \.C.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.G.a.m.e.C.o.r.e...i.n.i.
- And behold: the game loads from the DefaultGameCore.ini (DGC.INI) file in the "<Steam install path>\XComGame\Config\" directory, even if larger than the embedded version.
All the other INI files are read by default. Only these three have to be modded to load.
This means that no longer are BALANCE Mode settings the only ones taking effect from the text editable file in "\Config", and that additional items can be added to the DGC.INI file. Of the other files explicitly denied from loading, DefaultMPGame.ini (believed to be "multiplayer' related), and DefaultLoadouts.ini (initial equipment for classes of Soldiers and Aliens), are similarly enabled in the same manner.
NOTE: The Enemy Within (EW) expansion appears to still need this patch to read in the loose INI files. The original EU EXE definitely still does.
For instructions on how to search for the above filenames as unicode strings, with screenshots, see the wiki article How to DIY common mod solutions.
Bear in mind: Search for the following in hex mode for unicode values.
The following patches are taken from the documentation of the XCOMModHelper tool available at Github.com. That tool is recommended for Windows environments instead of manual edits.
Enemy Unknown patch 5 (changelist 382855).
Target Path: Binaries\Win32\XComGame.exeBinaries\Win32\XComGame.exe
Find:
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58 00 43 00
Replace:
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00
Enemy Unknown patch 6 (changelist 398179).
Same change, at offset 22535260 (decimal) / 0x157dc5c (hex).
Enemy Within patch 0 (initial release).
Target Path: XEW\Binaries\Win32\XComEW.exe
Find:
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58' 00 43 00
Replace:
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00
Enemy Within game file XComEW.EXE hex offset for Mac/OSX version: around 0x1FE7EB3.
Most likely reason for any failure to load the DGC.INI is that (assuming you have the EW version of the game) you modified the DGC.INI in the wrong folder tree. EU is under "XCom-Enemy-Unknown", while EW is under "XCom-Enemy-Unknown\XEW".
INI Loading Test
Try this to test that the loose DGC.INI file is loading:
- Edit the DGC.INI as follows:
- Replace the Original 'eItem_Shotgun' line 'Properties[1]=eWPAssault' entry with '=eWP_Anyclass' as shown below. You can just copy and paste this SINGLE line with any text editor. (Save your '.BAK' file to restore the original DGC.INI.)
- Load any previous save where you are in the XCOM base (the Strategy game phase) or play a new game to that point, go into the Barracks, View Soldiers, and change the Loadout of a Support class (which couldn't without this change and doesn't have any other class restrictions that might prevent it, as is the case with a Heavy). The 'vanilla' DGC.INI that's in the embedded 'Resource cache' only permits an Assault to equip a Shotgun. With this change, any class can that doesn't have other built-in restrictions.
- Anyone other than an Assault class equipping a Shotgun proves you are loading the DGC from the correct '\config' folder.
INI file size limitations
Increased Load Times
01 Oct 2013: The root cause of this problem has been identified as the format of all the "Package" entries in the [Engine.PackagesToFullyLoadForDLC] section of DLC's XComEngine.ini file. The 'dot' preceeding the keyword "Package" is a UE3 Special Character and needs to be replaced with a 'plus', as in:
[Engine.PackagesToFullyLoadForDLC]
{change}
.Package=Zhang_ANIM
{to}
+Package=Zhang_ANIM
Work is underway to alter the cleanup script below to incorporate this fix so it will only need to be run after a game update or verification of game cache files.
It's been determined (see the Forcing game to load from DefaultGameCore.ini thread) that DLC content is getting merged repeatedly whenever the game loads or restarts. This increases the size of the respective INI files, and causes the same DLC content to be reloaded multiple times per session. If the game has been loaded 100 times, those DLC assets are now being loaded 100 times. This is due to a bug in the process by which DLC content is merged.
This issue has been confirmed to exist with a vanilla, verified cache copy of the game without any mods.
The issue appears to be in the My Games folder XComEngine.ini / [Engine.PackagesToFullyLoadForDLC] section. In that section the following 37 lines are repeated <loaded> times:
MapName=Command1
Package=DLC_Day060
Package=Deco_Archangel0
Package=Deco_Carapace1
Package=Deco_Ghost0
Package=Deco_Kevlar1
Package=Deco_Psi0
Package=Deco_Skeleton1
Package=Deco_Titan0
Package=Helmet_Archangel0
Package=Helmet_Carapace1
Package=Helmet_Ghost0
Package=Helmet_Kevlar1
Package=Helmet_Psi0
Package=Helmet_Skeleton1
Package=Helmet_Titan0
Package=Head_AsnMale_F
Package=Hair_MaleHair_N
LoadForAllGameTypes=LoadForAllGameTypes
Package=Zhang_ANIM
MapName=CIN_LoadScreen
Package=UILibrary_MapImages_DLC1
MapName=DLC1_1_LowFriends
Package=DLC_Day060
MapName=DLC1_2_CnfndLight
Package=DLC_Day060
MapName=DLC1_3_Gangplank
Package=DLC_Day060
MapName=Command1
Package=Deco_Kevlar0
Package=Helmet_Kevlar0
Package=Deco_Skeleton0
Package=Helmet_Skeleton0
Package=Deco_Carapace0
Package=Helmet_Carapace0
Package=Hair_OriginalSoldier
The current workaround solution is to periodically remove the excess lines from the XComEngine.ini / [Engine.PackagesToFullyLoadForDLC] section. A VB script has been created to do this available here, but here are the instructions to recreate it for yourself if your antivirus program balks at it.
- Open notepad.
- Copy & Paste below text into notepad.
Const MY_DOCUMENTS = &H5&
Const ForReading = 1
Const ForWriting = 2
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("Shell.Application")
Set objFolder = objShell.Namespace(MY_DOCUMENTS)
Set objFolderItem = objFolder.Self
ConfigFolder = objFolderItem.Path & "\My Games\XCOM - Enemy Unknown\XComGame\Config\"
Set OrgFile = objFSO.OpenTextFile(ConfigFolder & "XComEngine.ini", ForReading)
strOrgText = OrgFile.ReadAll
OrgFile.Close
arrMapName = Split(strOrgText, "MapName=Command1")
if Ubound(arrMapName) >2 then
strOutput = arrMapName(0) & "MapName=Command1" & arrMapName(1) & "MapName=Command1" & arrMapName(Ubound(arrMapName))
timestamp = year(Now()) & "-" & Right(Cstr(month(now()) + 100),2) & "-" & Right(Cstr(day(now()) + 100),2) & "_" & Right(Cstr(hour(now()) + 100),2) & Right(Cstr(Minute(now()) + 100),2) & Right(Cstr(Second(now()) + 100),2)
objFSO.CopyFile ConfigFolder & "XComEngine.ini", ConfigFolder & "XComEngine.bak" & timestamp
Set destFile = objFSO.OpenTextFile(ConfigFolder & "XComEngine.ini", ForWriting)
destFile.WriteLine(strOutput)
destFile.Close
msgbox (Ubound(arrMapName)/2)-1 & " redundant sets of engine DLC packages has been removed from XComEngine.ini." & vbnewline & vbnewline & "A backup of your original file has been saved here:" & vbnewline & ConfigFolder & "XComEngine.bak" & timestamp
else
msgbox "Everything looks fine. No action taken."
end if
- File --> Save As... --> CleanUpXComEngineIni.vbs (make sure to not save it as CleanUpXComEngineIni.vbs.txt by mistake).
- NOTE: If your "My Games" folder is called something strange like "Meinen Spielen" you will have to change that on line 9.
When you run the script it will first check that <My Documents>\My Games\XCOM - Enemy Unknown\XComGame\Config\XComEngine.ini contains more than two "MapName=Command1". It only removes lines between the second and the last "MapName=Command1".
Enemy Within
This is a purchased expansion to XCOM:EU 2012, released in the USA on 12 Nov 2013. It installs to a XEW subfolder under the main XCOM install path. Most of the loose files from EU are replicated in this subfolder, which means EU and EW are basically separate games. There is some sharing of movie (.BIK) files. EW adds a launcher app XComLauncher.exe in the \XCom-Enemy-Unknown\XEW\Binaries\Win32 folder which is started by Steam and provides the option of which game (EU or EW) to launch.
The actual game executables this launcher runs are:
- \XCom-Enemy-Unknown\Binaries\Win32\XComGame.exe -- launches Enemy Unknown
- \XCom-Enemy-Unknown\XEW\Binaries\Win32\XComEW.exe -- launches Enemy Within
EW no longer makes a hash check of the compressed UPK files. So they can simply be decompressed and will be used after modification once the related .uncompressed_size files are removed.
However, the mod to force the game to use the loose XEW\XCOMGame\Config\DefaultGameCore.ini file is still needed. Hex edit the XComEW.EXE file as follows:
Find: (i.e. key is the 58 in the last line)
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58 00 43 00
Replace: (i.e. replace the 58 in the last line with 57.)
25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00
EW Update server (i.e. HOSTS file):
- 127.0.0.1 prod.xcom-ew.firaxis.com
- 127.0.0.1 65.118.245.139
These 'phone home' locations update the XComGamecore.INI file in config/loc on game launch, and are needed in addition to the entries for EU. However, early reports that disabling these connections cause the game to freeze are now attributed to other causes, such as not disabling 'auto-updates'. Cloud Synching of save game files still works with these addresses disabled. (While sometimes referred to as "the silent patching mechanism", so far there is no evidence that these servers are involved in actual changes to the game code. They still appear to only restore certain vanilla files involved in Multiplayer games, which can disrupt mods touching those same files.)
The EW 'phone home' addresses seem to only update XComGameCore.INI in My Games\XCOM - Enemy Within\XComGame\Logs\EMS\ at this time.
Extending/Replacing Functions
In early Dec 2013, wghost81 discovered it was possible to alter the UPK Package information so the game would utilize an expanded or replacement function of any size. This frees the mod creator from the constraint of keeping their code changes to within the original byte size limitations of the vanilla function, as well as providing the possibility to add new functions exclusive to your own mods. See the Hex editing UPK files article entry Hex editing III: Extending/Replacing Functions for details.
References
This information has been included (via Template:Recent Discoveries - XCOM:EU 2012) in the following articles:
