Reverse Engineering Unknown File Formats with ImHex
WerWolv, the developer of the open-source hex editor ImHex, demonstrates how to reverse engineer a custom binary file format from scratch. Using the save file from the game FEZ as an example, he walks through decompiling the game's C# code to locate the serialization logic, then explains how to write an ImHex pattern to decode the file's structure. The post covers essential techniques like identifying data types, handling strings and objects, and using pattern attributes, making it a practical guide for anyone tackling unknown file formats.
Without any more information, we’re basically stuck here. The data can mean anything, and only the program generating and parsing it can make sense of it.
- kstrauser
Story time: I worked the night shift at a motel during college. "Bored to tears" was an understatement. I was poking around the front desk computer system one night, as one does, and found its data file. Lacking anything else to do, I wrote a little hex dumper in BASIC so I could explore the file. The first thing I noted was that customer names were spaced exactly N bytes apart. Oooh, fixed records! Then I spent the next week or so reverse engineering the DB file format. I'd twiddle a field in a random customer record then look at their record in the hex dump to see what changed, then update my notes.
Eventually I wrote another little BASIC program to run analytics, like which customers spent the most with us, and auto-fix mistyped names ("This says 'Bb Jones'. Did you mean 'Bob Jones'?") by writing directly to the file.
I got a pretty sweet little bonus for my hackery.
- issung
I finally dipped into using ImHex when doing some reversing of the PS2 memory card format for https://ps2iodb.com.
The program is so good, and has enough of a reputation that I knew to reach for it even though I know nothing about reversing, and that was 3-4 years ago now! Also shoutout to the awesome GUI library is uses; https://github.com/ocornut/imgui
- truekonrads
My few rules are:
1) What's the entropy? This helps with encrypted/compressed. You probably need to overcome this?
2) What's the context? Apps are developed in context and context suggests formats. Is it C? Then expect structs. Does it need to go over the wire? Expect run-length encodings. Python/JS - JSON/pickle.
3) There are broadly speaking only a few ways you can read data back into memory: fixed layout, run-length encoding, terminators. A great case-study is ASN.1, once you've built an ASN.1 parser by hand no file format will ever daunt you.
- Noxwizard
I'm always on the lookout for a good hex editor. I've tried ImHex a few times and it doesn't quite fit the need I have and I prefer HxD for the simple things. The Pattern Language is pretty neat, I hadn't looked into that before. For the complicated things, it's not always a linear process like in the case of the blog article. Mainly, there's often lots of gaps in my structure until I figure them out. Some tools like ImHex or Kaitai, you can put in placeholder fields between two known objects, but that's kind of annoying. 010 Editor lets you highlight and put colored bookmarks, which kind of works. The workflow I've ended up using for reversing in-memory data structures is to get a hexdump of the memory, take a screenshot and then put it in OneNote. Then I can draw colored boxes or highlight and put annotations off on the side with what things are or notes. If a field is a memory address to another related and unknown structure, I often will then get a short dump of that memory and put a screenshot of it on the same page with an arrow to it. It's pretty kludgy, but I like the freeform nature of it. It's basically a RE notebook, just without any useful RE built-ins. If anyone has ideas or recommendations on better discovery workflows, I'd love to hear them.
- vivzkestrel
- stupid question: do you have any ideas how to go about doing this on a .unr file
- it comes from a game called splinter cell conviction made in custom unreal 2.5
- I have neither been able to get UE-Explorer or UEViewer to work with it
- I can send you the file, I just want to know how to change enemy AI spawn types on it