Finding Discord chats in OS X

After much searching the Discord cache folder is located here:

/Users/myusername/library/Application Support/discord

The cache folder follows the same file structure as the one found in Windows.

Discord cache folder in OS X

The following links will explain how to extract the json chat objects and how to convert them to html or xls files using a Python script. Although written originally for objects found in Windows the extraction and conversions steps apply all the same.

Extraction of json objects:

 https://abrignoni.blogspot.com/2018/03/finding-discord-app-chats-in-windows.html

Discord json chats conversion to html or xls.

https://github.com/abrignoni/Discord-JSON-chat-conversions

As background on how I located the correct folder I took the following steps:

1. Created a virtual OS X using Virtual Box. Virtual storage was VDMK format.
2. Installed the Discord program.
3. Logged in to my test account. The chats were synchronized from the ones in Discord servers.
4. Turned off the virtual machine. Created a snapshot.
5. Tried to process the snapshot with Autopsy. Wouldn't take it.
6. Made a clone of the machine in order to consolidate the snapshot and the image into one. Autopsy still wouldn't take it.
7. Installed Qemu. Converted the VMDK to RAW using the following command:
   
   quemu-img.exe convert -f vmdk 'J:\my-clone-mac-disk.vmdk' -O raw my-mac-disk.raw
8. Processed the raw file with Autopsy. In the keyword search section I added some of my test chats content and some other obvious terms like 'discord'.
9. Looked at the keyword search results. These lead me to the proper folder location described above.

At the end of the day I not only found what I was looking for but also learned about OS X folder structures and how to manipulate virtual machines. Everything you do is an opportunity to learn and share with others.

How to convert UNIX Epoch timestamps in SQLite DB fields to local time.

Short answer:
 select mychatfield, datetime(mytimefield/1000, 'unixepoch', 'localtime') from mytable

Detailed answer:
Most SQLite databases used in Android applications store their time stamps in UNIX time, also know as UNIX Epoch. Unix time defines a point in time as "the number of seconds that have elapsed since 00:00:00 Coordinated Universasl Time (UTC), Thursday, 1 January 1970, minus the number of leap seconds that have taken place since then." It goes without saying that these date are not stored in a way that makes them understandable to our consumers.

If your forensic tool does not parse a particular SQLite DB content automatically here is a quick way of converting those UNIX time stamps to local time.

For this example I will use an Tumblr Android SQLite DB that was extracted using Magnet Forensics Acquire and FTK Imager. To view the contents I used DB Browser for SQLite. The path location of the extracted database in my exemplar phone was:

userdata (ExtX)/Root/data/com.tumblr/databases/Tumblr.sqlite

Here is how the table looks with some sample data using DB Browser for SQLite.

Notice the timestamp field. A long string of numbers. Also notice the text field. The user content we are looking for if we are interested in chat content, for example. In order to present the chats with the corresponding time stamps in a human readable way we can use a SQL query to make the conversion.

Here is the query and the results:

select text as messages, datetime(timestamp/1000, 'unixepoch', 'localtime') as dates from messaging_message 

The datetime function takes three arguments in this example. The time string, UNIX epoch and local time modifiers. The reason the timestamp field values are divided by 1000 is due to the UNIX time being stored in milliseconds when the datetime function expects the UNIX time to be in seconds. By dividing we change the milliseconds to seconds.

In order to make the final column headers more descriptive change them in the query by using 'myfielname as newname' as seen above.

As always when doing these types of conversions on a case run validation tests with known data in a replica of the environment you are analyzing.

Discord JSON chats to XLS

Just added a script to convert the JSON chats to XLS spreadsheets.

Hopefully will be able to add batch functionality for all files in a folder as well as output selection all in one script by the end of the week.

• https://github.com/abrignoni/Discord-JSON-chat-conversions

@alexisbrignoni

Update 0:

Batch functionality and output selection done.

Update 1:

Added error handling via screen and error.txt file for chats that are unable to be converted to XLS.

Finding Discord app chats in Windows.

Discord on the desktop
In previous posts I discussed some ways of recovering and presenting Discord app chats from Android devices. This post will discuss how to find Discord chats in Windows machines and provide a simple way to visualize them.

As way of background Discord is a chat application whose target audience is people who play video games. Wikipedia states that the Discord app has 87 million unique users. With such a large user base it is interesting to find that current commercial tools do not parse Discord chat artifacts directly yet.

Location and Extraction
Like many Windows applications, the user generated files and configurations reside in the apps folder. For example:

C:\Users\SampleUser\AppData\Roaming\discord

The Discord AppData folder has the following structure:

Cache folder highlighted

The user activity files are located in the Cache folder. At first glance the chat files we are looking for are not immediately apparent.

Where are the chats?

A look at the Cache folder contents might seem familiar. It is the same file format of the Google Chrome cache located at

C:\Users\SampleUser\AppData\Local\Google\Chrome\User Data\Default\Cache

My default browser is Chrome and the Discord app uses the same storage structure. For comparison here is my Chrome cache folder view.

Same file structure

Since the file structures are the same it seem clear that the content we are looking for had to reside within the Chrome cache like folder structures in the Discord cache folder. Thankfully there are many tools that allow us to parse those structures and extract files from them. For this analysis I used the folowing tool:

ChromeCacheView v1.77 - Cache viewer for Google Chrome Web browser Copyright (c) 2008 - 2018 Nir Sofer

The tool will by default parse the Chrome cache at the default folder location in your computer. Just hit stop on the upper left corner and point the tool to the Discord cache folder. After processing the contents of the folder the tool will show something similar to the following:

Discord cache folder contents

The tool will allow you to print out all the metadata on screen to an HTML file. Even more useful is the ability to extract the actual files from the cache by selecting an entry in the list and pressing F4 (or via the menu.)

To find the chat files look in the URL column for addresses that end with "messages?limit=50".

messages?limit=50

These are the files we will export from the cache and will contain the chat messages. It is of note that they not always end in 50. In some cases they can end in 100. Also note that the name of these files is usually 50 as stated in the URL variable but that is no always the case. Some chat names start with the words After or Before followed by some sort of numeric ID. Hence the best way to identify them is to go by the ending of the URL column.

By clicking on file entry in the list one can see the pertinent metadata. It states that the files we are exporting are JSON files.

Content Type: application/json

Since they are JSON files they can be viewed in any regular file viewer. After exporting here is how one looks using Wordpad.

Content key is key

The formatting is hard to the eyes but understandable. Notice the content key, the value is the user generated chat. Each user generated block starts with the attachment key and continues with keys for multiple time stamps, user IDs, the chat content and the like.

In order to make it a little easier to read the values, the following script takes the json file contents and presents them as a collection of html  tables.

A little better

The best way to look at these files is to use Chrome since other browsers do not know how to decode certain characters, like emoji.

Look!!! Emojis!!!

The script is really simple and can be found here:

• https://github.com/abrignoni/Discord-JSON-chat-conversions

Script uses the json2html module that does the actual heavy lifting. It can be found here:

• https://github.com/softvar/json2html-flask

Pending will be the capability for the script to parse a group of chats in a folder instead of one by one.

I can be reached via twitter @alexisbrignoni

PD:

One can log into a target Discord account without the username and password by installing the Discord application on our forensic computer and then copying over the Discord app data folders from the examined computer. The program will require internet access for this to work. It goes without saying that consent to search or a search warrant is needed before attempting this type of access. 

-Brigs

Organization of American States - 37th Cybercrime Regional Workshop

Beautiful Guatemala City. 

What a great experience training and exchanging views with prosecutors from Costa Rica, the Dominican Republic, El Salvador, Guatemala, Honduras, Mexico and Panama in REMJA’s Cybercrime Regional Workshop.

Hope to be back soon...

Antigua Guatemala.
Worth visiting. Great local art and plenty of interesting history.

OEA flag and seals.

Welcoming ceremony. Event organized by the always
amazing @fiorella_mh

Lecture and case studies.

Cybercrime discussion with @TheJusticeDept prosecutors and
@OEA_Justicia subject matter experts.

@fiorella_mh

Honored to be able to speak on
mobile technology, social media, and dark markets

Dinner with faculty.

Viewing extracted Android app data using an emulator

In the previous blog posts I used free and open source forensic tools to view the content and file structure of the Android Discord app. After some testing and validation the presentation of the contents that the tools provide is understandable but not really intuitive nor user friendly.

What if you wanted to view the app content as the Android device presented it to the user without manipulating the device where the data originally came from? By using an Android emulator we can view the contents using the app itself as our viewer. Free tools will be used for this analysis.

Test Device

• Samsung SM-G530T Galaxy Grand Prime
• OS 5.1.1
• Rooted

 Tools

• Android Discord App with test data
• Magnet Forensics Acquire 2.0.3.8168
• Nox Player 5.0.0.1 - Android Emulator
• ADB - Android Debugging Bridge
• FTK Imager 3.4.2.6

Extraction

Using Acquire a physical source image of the phone is created.

Magnet Acquire Acquisition

With FTK imager the following folders are exported from the source image:

• /app/com.discord-1
• /data/com.discord

The app data folder contains the Discord application itself. It is named "base.apk".

Discord APK - base.apk

 The data folder has all the content related to the app. Be aware that different apps will require different folders to be identified and exported. For example some apps use the SD card to store data needed for the app to work properly. Using a test device, identify all the folders needed by the app then export them from the source image.

Discord Data Folder

After exporting the folders start the Nox emulator. Depending on the app you are working with make sure to change the settings on the emulator. By default the emulator starts in tablet mode.

Tablet Mode

If the app being worked on is from a phone, change the mode and resolution accordingly.

Change Mode & Resolution

Since I am using a test phone, here is how the emulator looks after the change.



Leave your emulator running since we will connect to it using ADB.

ADB - Connecting and pushing extracted data to the emulator 

We will use ADB to connect to the emulator and push into it the data repositories we identified previously. Nox comes with ADB already available in the main installation folder. For a default installation the Nox ADB will be found at C:\Program Files(x86)\bin\adb.exe or adb_nox.exe.

Open a command prompt at the previous path and connect to the emulator using the following command:

• adb connect 127.0.0.1:62001

To test connectivity use the following command:

• adb devices

If successful you should see the following on your screen.

Connected to Emulator

In another command prompt open a shell to the emulator using ADB.

• adb shell

ADB Shell

The first command prompt will be used to push/upload the apk. The second command prompt will enable us to interact with the emulator's file system.

Use the following command, in the first command prompt, to push the apk into the emulator into the corresponding directory:

• adb push base.apk /data/app

APK to Emulator

The application will appear on the home screen of the emulator. Press on it to initialize it. By starting the apk, the folder structures needed by the app will be created in the emulator.

After seeing the initial screen, press the Recent Taks button in the emulator (the square above the >> on the lower right side of the screen) and swipe away the app. 

Before we push the app data to the emulator we need to delete the folders that were created by the app when we initialized it.

On the second command prompt, the one with the ADB shell, navigate to the following folder:

• /data/data/com.discord

There you will see all the folders that were created by initialization. Use the following command to delete them:

• rm -r *

Delete App Folders

Go back to the first command prompt and push the extracted data app to the folder we just emptied.

• adb push com.discord /data/data/com.discord

Extracted App Data pushed to Emulator

If the push was successful the following will be seen on the prompt:

Successful Push

On your ADB shell prompt you will now see that the folders extracted from the image are now located within the emulator.

Successful Push - Folder Structure

To summarize so far we have:

1. Made a physical source image from the target device.
2. Extracted the target app and app folders (Discord.)
3. Push the extracted app to the emulator and initialize it.
4. Close the app and delete the app created folders.
5. Push the extracted app folders and their content to the emulator.

Go to the home page on the emulator and press the Discord app. All the content of the app can now be viewed in the emulator.

Discord Direct Message

Considerations

This method of review is really useful for showing the extracted data to non technical people. It is a great visual aid to see the data as the original user would have seen it herself. 

When doing this type of analysis we need to always adhere to best practices. For this example my workstation and emulator were connected to the internet. Notice the green dot next to the username at the bottom of the following image.

Logged In

Make sure your workstation is not connected to the internet since some applications will reach out and log in to the servers that the app uses. This could cause an issue where the analyst might inadvertently conduct a search/download of data from the server without proper legal authorization. 

Be aware that emulator analysis like this one might not work with applications that are closely integrated with the operating system, e.g. Google apps.

In my testing I have found that by closing the app or shutting down the emulator, the pushed data is no longer accessible and the app returns an invalid install error message. Make sure to take screenshots or video capture the pertinent screens before you close the app or shutdown the emulator. To access the app and data after shutdown will require reinstalling Nox and redoing the whole procedure again.

Discord App - Missing values not missing

* Oct 17, 2018 - Update with further insight here. *

Last post I did a quick overview of the Discord app for Android. At the time I commented that "both the messages and the usernames are missing the last letter as represented by the tools." Upon further review I figured out that
the alphabetic value of the last hex number at the end of every sentence can be decoded starting at E1h and ending at FAh.

For example the first 5 alphabetic values would be:

E1h = a
E2h = b
E3h = c
E4h = d
E5h = e

One has to only continue to map letters to hex values sequentially until the whole alphabet is represented.

By looking at the hex at the end of the sentence and comparing it to the corresponding alphabetic value one can figure out what is that last letter the tools used did not show us.

Go figure.

Discord Android App Review - DFIR

What is Discord?

Discord is a communication platform for video gamers. It advertises minimal CPU usage, high voice quality and multiple social media features like friend lists. As part of their communication platform, Discord has corresponding iOS and Android apps.

As a gaming enthusiast myself (Broodwar & SC2 forever!) and DFIR practitioner, there was plenty of motivation to see how this app stores user generated data in Android.

Test Devices and User Generated Data

The following testing devices were used:

• Samsung GSM SM-G530T Galaxy Grand Prime 
• OS 5.1.1
• Privileged Access (root): Yes 
• Samsung SGH-I747 Samsung Galaxy S3
• OS 4.1.2
• Privileged access (root): Yes

Using the same wireless network connection, the Discord app was downloaded and installed on the test phones. Two Discord accounts were created, one for each phone.

To create data for the app to store, I sent the following messages and pictures from one phone to the other using Discord. I also made a short VOIP call in the same manner. Oh and emoticons, of course.

App Interactions

Acquisition Tools

For this analysis I wanted both free and paid for tools.  It goes without saying that paid for tools come with customer support and ease of use that might not be available on a free or open source tool. Still a lack of funding for tools shouldn't be an obstacle to engage in meaningful forensic case work. Ideally an examiner will leverage all the tools at her disposal to paint the most complete picture possible.

 The following software was used for forensic imaging:

• Cellebrite UFED4PC 
• Magnet Forensics Acquire 2.0.1

Magnet Acquire Acquisition

Processing Tools

In order to read the imaged file system/s and for the parsing of artifacts the following tools were used:

• Cellebrite Physical Analyzer (PA)
• Autopsy 4
• FTK Imager 3.4.2.2

Both PA and Autopsy present the file system and parsed artifacts on the left of the screen and the related data on the panes to the right.

Analysis

PA presents the parsed chat artifacts it knows how to handle automatically under the 'Analyzed Data'  section on the left pane of the PA screen. No chats were automatically parsed for Discord. 

With PA it is always good to check the Installed Apps section. One is able to tell which apps are parsed by Cellebrite and which ones are not. Notice that next to the Discord line, under the 'Decoded by' heading there is no entry. If it were parsed by PA it would say 'Cellebrite' and a corresponding entry would have been found under the Chats section. Conveniently the app description itself tells us it is a chat application. The entries in the Installed apps section are pulled by PA from the 'localappstate.db' and the 'AndroidManifest.xml' files as seen in the screenshot below.

Physical Analyzer - Installed Applications

By contrast Autopsy does not parse the installed applications in order to present it in a spreadsheet as PA does. A manual search of the file system is needed to identify apps and data stores of interest. In most cases Android apps' data stores can be found in the 'userdata(ExtX)/Root/data/' directory. That being said, Autopsy does parse text messages and many other Android artifacts. In this particular instance both PA and Autopsy do not parse Discord artifacts automatically. The lesson to be learned is the value of manually traversing the acquired file system to make sure nothing is being missed no matter what tool is being used.

Within the data stores a directory for Discord was found.

Physical Analyzer - Discord App File Structure

Within the 'userdata (ExtX)/Root/data/com.discord/files' directory multiple files with the 'PREF_K' prefix were located. For this analysis we will review the following:

PREF_K_STORE_MESSAGES_CACHE_V15
PREF_K_STORE_USERS_MAP_V7
PREF_K_STORE_USERS_ME_V7,


The PREF_K_STORE_MESSAGES_CACHE_V15 file contained the test messages that were sent.

Autopsy 4 - Indexed Text View

It seems the file, which has no extension, calls some java functions first then below it stores the messages. In my experience most chat apps keep their messages in sqlite databases. In this instance no sqlite databases were found within the Discord directory. It would seem to me that these java related files are the data stores for the app itself. This assumption does require further validation. Note that both the messages and the usernames are missing the last letter as presented by the tools. For example the username 'spotmusic' is presented  in the indexed text view as 'spotmusi' and the 'hello spot' message is presented as 'hello spo'.

For this test three pictures were sent. Two URL paths for every sent picture are contained within the message data store file. Here is an example.

Discord - Shared Media Paths

I was able to access the pictures by copy-pasting the URL on my computer browser. After 5 hours of the pictures being sent, the URL still works. In addition to the picture URLs, Discord keeps the send/received picture thumbnails in the
'userdata (ExtX)/Root/data/com.discord/cache/app_images_cache_small/v2.ols100.1folder.'
The images have an internal naming convention and end with the '.cnt' extension. Autopsy recognizes them under the 'Extension Mismatch Detected' section under 'Extracted Content'.

Autopsy 4 - Extension Mismatch Detected

Saved Discord received images go to the default downloads folder for the device. The file keeps the original file name.

By looking at the data store file, patterns seem to emerge. Further validation is still required.

For messages the following entries appear in the following order:

• Username missing the last letter.
• Message missing the last letter. 
• Time of message. 

For a VOIP call:

• Username missing the last letter.
• Call header
• Time
• Username missing last letter
• Time

FTK Imager - Call entry

For an attachment:

• Attachment header (for the first image)
• Attachment name missing last letter
• URL #1
• URL #2
• User
• Time value

FTK Imager - Attachments

For a pinned messages:

• Username missing last letter
• Unknown value, might denote pinned message
• Time value

FTK Imager - Pinned Message

It is of note that the emoji hex values can be identified in testing by sending the same emoji multiple times and looking for repeating patterns in the message portions of the file.

The PREF_K_STORE_USERS_MAP_V7 file denotes the individuals involved in the chat.

Autopsy 4 - Users Map

The PREF_K_STORE_USERS_ME_V7 stores the username of the logged in user.

Autopsy 4 - Users_ME

Conclusions

Truly no one tool can provide by itself a complete picture of what user activity transpired in a particular device of interest. Further testing and validation can reveal additional artifacts of interest regarding this particular app. There are estimates that indicate that Discord has approximately 25 million users.

By using paid for commercial tools as well as open source ones, the point is made that progress can be made regarding case work without the need of a bottomless budget. 

-Brigs
@AlexisBrignoni

"Hello World"

Hi and thanks for stumbling into my blog. Here I will focus on Information Security, Digital Forensics and Incident Response (DFIR) topics. Hopefully the upcoming content will be useful or at least amusing.

DFIR is a fast moving field and to be current you have to not only learn from others but also try and humbly give back. This is my first effort at doing so online. I hope to hear/share some ideas on:

1) Workflow management. Work smarter not harder. Are there any procedures that take too many steps that you wish took less?

I look to identify workflow issues like those and make them better. Same with with scale. Can a procedure be developed that applies to multiple pieces of evidence or artifacts? If so, lets hear it!

2) Presentation skills. I use Prezi to teach introductory networking and cyber topics. Will share some of those here. What does it take to be a good presenter on tech topics beyond the slides?

3) Programming. What are the most useful programming/scripting languages used for DFIR?

4) Learning. The best way of learning something is to explain it to someone else. Will try to do so here as well.

 -Brigs