Until not too long ago extracting data for forensic analysis from Chromebooks seemed impossible. Thanks Daniel Dickerman's workflow we can extract data provided you have a username and password for the device.
Thanks to Magnet Forensics the process has been automated and now its implementation is available as a free software tool called the Magnet Chromebook Acquisition Assistant.
Currently CLEAPP parses 38 artifact categories. The project wouldn't be what it is without the contributions from Alex Caithness and Ryan Benson. Thank you so much!
Thank you gentlepeople <3
Installation
If you are familiar with how iLEAPP of ALEAPP works then you already know how to use CLEAPP. These projects are done in python. If you are not familiar with how to run python scripts just follow the steps in the following video.
Run the cleappGUI.py script for the graphical user interface version. It will look like this:
Click around and done
Notice the list of modules on the left. You can parse all or select individual modules. CLEAPP is pretty fast so for most purposes running with all modules enabled is recommended.
Here is a short list of some modules it supports:
Chromebook device details
Chromebook device logs
Chromium Browsers
Instagram Threads
Chromium LevelDB data stores (Thanks Alex Caithness & Ryan Benson)
Microsoft RDP
Real VNC
Google Docs
and tons more...
After CLEAPP finishes processing the output will be in the following formats:
HTML report
Tab separated values text files for every artifact
KML files for artifacts that have geolocation data points
SQLite timeline file for artifacts that have timestamps
SQlite contacts file for artifacts that have contacts information
The HTML report contains the categories and artifacts on the left of the report.
HTML report
The Device Details tab will have information on the Chromebook like serial number, current operatin system version, and more.
Device Details
One of the interesting facts about Chromebooks is that they can run Android apps. As time permits I plan to merge all ALEAPP artifacts for use in CLEAPP and make sure that both projects support Android artifacts.
Since this is a community project we will be more than happy to have additional collaborators.
Use one of the following files to determine the Android version of a digital forensic extraction that is lacking the \system\build.prop file:
\data\system\usagestats\0\version
\data\system_ce\0\usagestats\version
Long version
Most automated tools that parse Android full file system extractions depend on the /system/build.prop file to determine the Android version among other device identifiers. Due to how variable are Android implementations as well as data extracting software a build.prop file might not be available. Is there a way to determine the Android version of an extraction by only looking at the userdata directory? The answer is yes. This was useful to me since some of my digital forensics tooling for Android extractions would benefit from programmatically identifying the Android version when a build.props file is not available.
One of the ways Android devices keep track of application activity is by registering events in the usagestats directory. Depending on the Android version these can be kept in XML or protobuf format. These can be found in the following locations depending on your Android version:
The usagestats folder contains a plain text file called version. It is usually composed of two lines. The first one being a number and the second one a series of alphanumeric values separated by semicolons.
Usagestats/version file from a Samsung SM-G960U
The filename clearly indicated that the content had to be versioning related. Since the build.props file is well understood and documented I made a comparison between the too to try and determine the provenance of the version's file content if possible.
Notice in the following image the side by side comparison and color coding of similar values of files extracted from a Samsung device.
From the version file's second line we can determine the following:
First items = Version release = Android version
Second item = Codename = Fingerprint
Third item = Incremental build version
Fourth item = CRC = Country Specific Code
Thanks to Kevin Pagano (@KevinPagano3) for identifying that the fourth value is a CRC and for leading me a list that matched codes with values. These CRC values seem to be Samsung specific.
It is of note that Pixel devices have less values within the version file. There is no CRC and no final value. Still the Android version was the same as the one located in the build.props file. This was true across all sample extractions I was able to check.
Version file from a Pixel phone
What about the number on the first line? Through the process of testing the values on the second line a pattern appeared for the number in the first line. It is as follows:
3 = Android 8, 9
4 = Android 10
5 = Android 11
Jessica Hyde (@B1N2H3X) suggested I take into consideration how the file would look, if anything, after an operating system upgrade. Great point! Thanks to Josh Hickman (@josh_hickman1) that was really easy to do. He has well documented test Android images for community use and testing. By looking at the values within the version file on his Android 10 extraction and then the values on the upgraded Android 11 image I was able to determine that an update would produce an additional file in the directory where version resides called migrated.
The migrated file.
The migrated file contains the number in the first line of the version file previous to the upgrade. After the upgrade the version file contains the numbers that are consistent with the current version. The next image is the value contained within the migrated file in the Android 10 extraction.
Migrated file with a value of 4
Now compare it with the values within the version file in the same Android 11 extraction.
Version file with a value of 5
This behaviour was also confirmed by Carlos Eduardo (@GalloDu) with his own upgraded device data.
Confirmation of migrated & version relationship
Based on this behaviour it follows that the presence of a migrated file indicates a major operating system upgrade. By comparing the contents of the migrated and version files the analys can determine from and to what version the device was upgraded to.
Pending work
For this analysis I only had access to Samsung and Pixel files. It would be of use if migrated and version files from other vendors (LG, OnePlus, etc...) are shared to see how they might defer and/or what additional data they might provide if any.
Implementation
I have made a parser for the version file within ALEAPP. The script will identify and use Android version number contained in either the build.props or version file for reporting and artifact purposes. To quickly view the data press the device details tab at ALEAPP's report home page.
Device details tab with Android version data
As always, I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.
For the last couple of days I've been working creating a parser of Discord JSON chat files using iLEAPP. If you are not familiar with iLEAPP it is a Python 3 framework designed to parse useful forensic artifacts from iOS devices. More on iLEAPP here. I wanted to validate some findings on a case I am working with the amazing @i_am_the_gia and as part of the process I used the newly created parser on @Josh_Hickman1 excellent iOS testing images. You can get his testing images here.
Here is iLEAPP's HTML report for the chat:
Here is the output for the Discord user's email and user ID:
In that same moment I watched the most amazing trailer for The Mandalorian Season #2 thanks to @KevinPagano3. As you all should know by now, the Child just steals every scene with just how cute it is.
Going back to my report I copy one of the URLs in the attachment column and pasted it into an internet connected browser to see if it would come up. In past (2017) I did some testing on Discord for Android and found out that the links in chats could be copy-paste into a browser and be accessible from anywhere by anyone.
With Josh's image I confirmed that was still the case. And what did the URL image in the chat contain?
Coincidence? I think not. :-D
As always, I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.
Search for user_id_cache and email_cache It's only the user id, and not the username. Search the messages in the cache.db (iOS) or 50.json (Windows) to match up the userid with the username.
Thank you so much TheKateCain. Super useful information!
Greetings! Below is a list of assignments from recent classes.
Reminder:Assignments listed below indicate what to complete before class; make sure that you are signed in to Discord in order to access the practice files
🐍
Class 10 on 06/25/2020
No homework / study hall
Class 11 on 06/30/2020
No homework / study hall
Class 12 on 07/02/2020
Conduct online research of argparse and make a script that takes two arguments and prints them to screen
Interested in learning Python? Here's the syllabus from our DFIR Python Study Group course. Follow along by getting the book, doing the homework, and watching the YouTube videos.
Note: Assignments listed below indicate what to complete before class; make sure that you are signed in to Discord in order to access the exercises via the links
Textbook:Head First Python: A Brain-Friendly Guide, 2nd edition
🐍
Class 0 on 05/21/2020
Ch. 1 pp. 1-19 until “What We Already Know”
Class 1 on 05/26/2020
Ch. 1 pp. 20-46 until “Chapter 1’s Code”
Class 2 on 05/28/2020
Ch. 2 pp. 47-55 until “Creating Lists Literally”
Class 3 on 06/02/2020
Ch. 2 pp. 56-94 until “Chapter 2’s Code, 2 of 2” / blank page
Calls a function that selects timestamp, partner_jid, and body from the messagesTable in the Kik database; prints them to screen; and calls a function to generate a text file report
Calls a function that extracts the timestamp, author, and content from the Discord JSON file; prints them to screen; and calls a function to generate a text file report
Class 10 on 06/25/2020
No homework / study hall
Class 11 on 06/30/2020
No homework / study hall
Class 12 on 07/02/2020
Research information online about argparse
Research information online about dunders for name and main
Make a script that takes two arguments and prints them to screen
Even though iOS full file system extractions are fairly commonplace the need to parse iTunes backups has not subsided. In many situations the only piece of evidence available could be an iTunes backup located in an external drive or one acquired from an iCloud data set. Most forensic tools are designed to parse these backups for relevant artifacts. What would happen if we took these backups and fed it to our tools in the same file structure they would be in if the backup had been restored to the iOS device? In other words what would our tools do if the backup file path structure was normalize to the file path structure found within the iOS that created them?
Video version
iTunes Backups - A super, ultra, mega short primer.
For those not familiar with iTunes backups I highly recommend reading Jack Farley's blog post on the topic. It can be found here:
An iTunes backup takes files from the device, renames them, and puts them in different folders where the key to recreate the almost original file structure is to look at the contents of the manifest.db database that is also created at backup time. I say almost because instead of keeping track of the whole path as it would have been on the device it abstract part of it under a series of category names called Domains. I think a comparison of paths would be a better way of understanding such transformations.
Notice how the farther away we are from the path as found on the device the more abstract it becomes. I bet you are wondering how could I tell that the WirelessDomain stand for /var/wireless within the file system. The domain to path is contained within the device in the following location:
System/Backup/Locations/Domain.plist.
The plist is not part of the backup and can only be accessed from an iOS full file system extraction. In my testing the domains seem to be the same across all iOS devices. Again, highly recommend reading the previous link to understand the how and the why of these transformations.
If you are interested in recreating an iTunes backup as defined in the manifest.db, using the domains, check out Jack Farley's python script here:
I can't count how many times I have parsed an iTunes backup to realize the following:
- Not all relevant items are parsed.
- For items that are parsed the file path provided is the one as created by the backup. This tells me nothing in regards to where it was the file originally located on the phone. I don't like having to query the manifest.db data store by hand one bit thank you very much.
What if we normalized the backups to the full path as found within the device and then have our tools parse it? Would it possibly parse more if it believed it was a full iOS file system extraction?
Edward Greybeard (great pseudonym) made a script for easy normalization of these backups in Python. You can get it here:
For my testing I parsed an iTunes backup two times.
First run as a file system using domains.
Second run with fully normalized iOS paths.
The tool I used was my own, iLEAPP. I ran a limited set of artifacts for this example. The artifacts searched for where the same in both runs. You can find it here:
It is one thing to use a tool that has not been explicitly designed to parse iTunes backups and have it show this type of results. Would the same happen with a commercial tool? The answer, as everything in forensics, is it depends. I have done this type of double runs with commercial tools and found that they either parsed more artifacts, some times less, and sometimes both less and more. If the last one doesn't make sense think of a report that parsed less artifacts overall but included a few that weren't in the first run.
Takeaways
Is this a type of analysis you should do on every iTunes backup in every case? No. Commercially available digital forensic tools do a great job as is. But if the case hangs on that one iTunes backup it doesn't hurt to try different approaches.
In regards to vendors my hope is that when parsing iTunes backups they can add the fully normalized path for the artifact as part of the metadata panes in their tool user interface. Normalized paths for all backup files. The analytical benefit you can get from knowing where on the device was a file originally located can be incalculable.
As always, I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.
Make no mistake, M.E.A.T. had me at hello. iOS extraction open source python software by Jack Farley (@JackFarley248). It creates a file system extraction of jailbroken iOS devices using Apple File Conduit 2.
I love Twitter but sometimes the algorithm, or just plain bad timing, has me miss some of the most juicy tweets and announcements. Thanks to Phill Moore (@phillmoore) I was made aware of the newly release Mobile Evidence Acquisition Toolkit (M.E.A.T.) by Jack Farley. I've been following his work and using his code for a while now. I cannot encourage you more to give his scripts a shot. From a learning/training use to a data acquisition and validation perspective, these tools are worth your time.
M.E.A.T. will give you a full file system extraction from iOS devices with a single command. If you had the pleasure (or pain) of extracting a full file system over WiFi using SSH you will appreciate the speed and simplicity of this USB based method. The following is a quick guide and review on the scripts and how to execute them.
Pre-requisites & download
Go to the github repo for M.E.A.T and download the scripts. As stated in the repo's readme you will need a Windows machine with Python 3.7.4 or 3.7.2 installed. Before running the scripts make sure you go unzip the downloaded file and run the following command from the root directory of the scripts.
pip install -r requirements.txt
This will make sure all the dependencies are installed. Also your target iOS device will need to be jailbroken and have Apple File Conduit 2 (AFC2) installed via Cydia.
A great guide on what jailbreaking entails with emphasis on the Checkra1n implementation of Checkma8 see Ian Whiffin & Shafik G. Punja blog post here.
Assuming the target device is jailbroken with AFC2 and your device has Python with all proper dependencies you are ready to go.
Let's beat it
Connect your iOS device to your computer. As expected make sure to select Trust Device at the prompt on your iOS device and provide the proper pin/code/pass as needed. With trust established we can connect to the phone.
Navigate to the script's root directory. It will look like this:
Open a command line interface at this location and run the following command to examine the help documentation.
python MEAT.py -h
You will see the following. Pretty self-explanatory.
Sick MEAT ascii art. I think it is cured by now.
As seen in the help you can generate MD5 and SHA1 hashes for your extracted files. Delicious!
For this example I will run a file system extraction that will pull everything from the root of the device. The logical option will only extract data from the \private\var\mobile\Media directory. Be aware that the -v option will add some additional time to the extraction since you will be getting a lot of output sent to the screen.
Before starting the extraction I create the output folder in the same directory from where I am running the script. You don't have to do this of course. I do because it shortens how long the extraction command will be. In this example I typed the following to create my output directory:
mkdir output
To start the extraction process in verbose mode without hashing type this:
python MEAT.py -iOS -filesystem -o ./output -v
At the start of the process you will see your target device information.
Since we are running in verbose mode you will see a flying matrix movie like screen of text on screen.
These are files being extracted. `The process on my device took around 90 minutes for a 15 GB extraction in verbose mode. When done you will have an iOS file system folder and a log. If you selected the hashing option then you would have seen a csv file with all the calculated hashes.
Be aware that you might find some difficulty running scripts on some older iOS devices. This is OK. No software executes a 100% of the time. That is to be expected as well as these scripts getting better with time. The fact that even though these were released a few days ago you are able to extract so much from so many iOS devices is amazing.
Here is the contents of the iOS-Filesystem directory.
Now what?
iLEAPP for M.E.A.T.
With the extraction done you can parse it with your favorite digital forensics commercial tools. In order to keep the open source vibe going I will parse it with my own tool, the iOS Logs Events And Properties Parser (iLEAPP), and see what we can get. You can get iLEAPP here:
As expected the tool parses the artifacts and provides a report. Currently I am updating the reporting function in iLEAPP so it has searching by report section and an overall more polished look. Credit goes to Yogesh Khatri (@swiftforensics) for his work on the reporting features in ALEAPP that I am now porting to iLEAPP.
iLEAPP command line execution:
iLEAPP reporting from M.E.A.T. extraction:
Conclusion:
We are truly living in the golden age of mobile digital forensics in the midst of a vibrant community of practitioners that work together to make the industry, tools, and knowledge more useful and accessible. Again, a big thank you to Jack Farley for his work on this tool. It is greatly appreciated.
As always, I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.
For details on how to jailbreak an iOS device see here: https://www.doubleblak.com/blogPosts.php?id=12. Lots of detail on how to use Checkra1n so a full file system data dump can be extracted for analysis.
Due to Covid-19, and the fact that social interactions in person have been limited because of it, a slew of group video chat applications have taken off in popularity. One of those is Houseparty for all major operating systems. This post will deal with the iOS version of the app.
For this analysis I used the excellente public test image created by Josh Hickman (@josh_hickman1). His images have detailed documentation regarding what apps were used, what user activity was generated, and when. This process is key when dealing with an unknown app or one that is not parse by commercial tools. You can get these excellent test images here:
In order to investigate a non-parsed app the process I recommend is to generate a known data set collection. That way one is aware of what to look for while trying to decipher how is the data stored. In this case Josh's image, since it is so well documented, will serve as our research platform.
Our test image has the following documented activity:
This is the data we will be searching for in our app data store. The first step is to locate the app data folder in the iOS full file system extraction. To do this I ran the extraction on iLEAPP. This is a collection of python 3 scripts designed to extract interesting artifacts from iOS images. You can download iLEAPP here:
After processing a report is generated. For simplicity I limited the report to the applicationstate.db artifact. This is the database that iOS uses to keep track of what apps are installed and where.
Using the search feature in the report I was able to locate the app and the location where the user generated activity is kept. If you are not sure what the bundle ID of the app is you can easily find it here:
The path to follow is under the Sandbox Path column. Notice how app directories in iOS are identified by a long GUID number. This is why querying the applicationstate.db is so important. It is the fastest way to determine what GUID name directory corresponds to the app of interest.
After arriving to the target directory we find the usual app structure for iOS apps.
Inside the documents folder is our data store of interest. A Real file named houseparty.rocky.realm.
In order to view the contents of this data store one has to have Realm Studio installed on our analysis computer. Real Studio can be found here:
After opening the data store three classes are of interest. The first one is RealNote. This one contains the expected chats with recipient IDs and timestamps.
The second one is RealmPublicUser. This class contains information about the message recipients.
The third one is RealLocalContact. It has additional information of the local user account for the app.
One way of reporting the contents of these data stores is to export the contents to JSON.
With the data in JSON format one can extract whatever classes are needed for reporting purposes.
A quick triage way to visualize the data without the needing Real Studio is to process the exported JSON file through a JSON to HTML converter. One can be found here:
This conversion helps, in my opinion, the user see delimiting lines between keys and values more easily.
Conclusion
Realm databases are becoming more prevalent in mobile analysis. We will be well served in practicing how to approach these new data stores. I believe they could possibly replace SQLite databases in the future.
As always, I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.
